-- Logs begin at Fri 2022-06-17 06:34:13 UTC, end at Fri 2022-06-17 07:38:13 UTC. -- Jun 17 07:05:36.168043 dv-cinder-834822-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 17 07:05:36.175348 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 17 07:05:36 2022] *** Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: nodename: dv-cinder-834822-4-iscsi Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: machine: x86_64 Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: clock source: unix Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: pcre jit disabled Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: detected number of CPU cores: 4 Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: current working directory: / Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: detected binary path: /usr/bin/uwsgi-core Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: your processes number limit is 47680 Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: your memory page size is 4096 bytes Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: detected max file descriptor number: 2048 Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: lock engine: pthread robust mutexes Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: thunder lock: enabled Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 17 07:05:36.179089 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 17 07:05:36.223540 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: Python main interpreter initialized at 0x55fdac4289c0 Jun 17 07:05:36.223540 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: python threads support enabled Jun 17 07:05:36.223540 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: your server socket listen backlog is limited to 100 connections Jun 17 07:05:36.223540 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: your mercy for graceful operations on workers is 90 seconds Jun 17 07:05:36.223919 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: mapped 403077 bytes (393 KB) for 2 cores Jun 17 07:05:36.223995 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: *** Operational MODE: preforking *** Jun 17 07:05:36.224294 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: *** uWSGI is running in multiple interpreter mode *** Jun 17 07:05:36.224294 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: spawned uWSGI master process (pid: 80107) Jun 17 07:05:36.224495 dv-cinder-834822-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 17 07:05:36.225879 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: spawned uWSGI worker 1 (pid: 80117, cores: 1) Jun 17 07:05:36.227009 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: spawned uWSGI worker 2 (pid: 80118, cores: 1) Jun 17 07:05:36.227009 dv-cinder-834822-4-iscsi devstack@keystone.service[80107]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 17 07:05:41.702713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 07:05:41.702713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: from cryptography.utils import int_from_bytes Jun 17 07:05:41.703473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 07:05:41.703473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: from cryptography.utils import int_from_bytes Jun 17 07:05:41.703882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 07:05:41.703882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: from cryptography.utils import int_from_bytes Jun 17 07:05:41.703990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 07:05:41.703990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: from cryptography.utils import int_from_bytes Jun 17 07:05:41.752564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 17 07:05:41.752760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 17 07:05:41.752854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 17 07:05:41.752941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 17 07:05:41.753014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] command line args: [] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 17 07:05:41.753140 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] command line args: [] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 17 07:05:41.753232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 17 07:05:41.753378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ================================================================================ {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 17 07:05:41.753456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 17 07:05:41.753582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ================================================================================ {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 17 07:05:41.753666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] admin_token = **** {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.753841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] admin_token = **** {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.753932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] config_dir = [] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.754040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.754125 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] config_dir = [] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.754223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] config_source = [] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.754307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.754404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] debug = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.754494 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] config_source = [] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.754606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.754867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] debug = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.754867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.755031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.755114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.755186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.755254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.755381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.755476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.755633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.755722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.755867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] list_limit = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.755959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] list_limit = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.756036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] log_config_append = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.756155 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] log_config_append = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.756235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.756312 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.756540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] log_dir = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.756673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] log_dir = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.756778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] log_file = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.756886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] log_file = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.756988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.757084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.757184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.757284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.757386 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.757492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.757616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.757734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.757850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.757926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.758050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.758134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.758207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.758283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.758353 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.758419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.759276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.759409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.759520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.759634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.759709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.759774 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.759851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.759939 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.846329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.846547 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.846627 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.846696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.846921 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.847011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.847455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.847534 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.847655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] publish_errors = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.847734 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] publish_errors = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.847905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.847983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.848176 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.848255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.848367 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.848444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.848629 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.848732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.848880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.848958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.849126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] standard_threads = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.849203 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] standard_threads = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.849369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.849518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.849626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.849704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.849808 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.849885 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.849993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] use_journal = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.850070 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] use_journal = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.850177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] use_json = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.850253 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] use_json = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.850359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] use_stderr = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.850435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] use_stderr = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.850538 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] use_syslog = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.850616 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] use_syslog = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.850779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.850861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 07:05:41.851049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.851125 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.851297 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.851375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.851568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.851644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.851817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.851892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.852091 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.852172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.852339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.852415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.852598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.852673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.852844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.852932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.853100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.853176 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.853353 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.853430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.853615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.853694 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.853857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.853940 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.854108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.854191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.854351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.854436 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.854602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.854678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.854857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.854933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.855133 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.855212 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.855395 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.855472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.855655 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.855731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.855908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.855986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.856174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.856255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.856436 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.856511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.856705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.856784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.856968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.857093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.857208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.857319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.857440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.857552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.857664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.857776 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.857887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.858008 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.858122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.858233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.858351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.858470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.858589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.858710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.858824 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.858936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.859047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.859164 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.859283 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.859404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.859523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.859648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.859768 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.859881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.859993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.860133 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.860255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.860368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.860486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.860599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.860709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.860902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.861017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.861128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.861239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.861351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.861462 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.861581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.861694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.861805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.861918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.862037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.862150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.862273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.862387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.862498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.862609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.862720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.862838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.862960 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.863073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.863184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.863280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.863379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.863473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.863577 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.863673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.863766 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.863860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.863955 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.864109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.864240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.864343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.864438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.864579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.864694 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.864808 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.864932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.865038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.865149 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.865283 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.865402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.865524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.865628 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.865748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.865869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.865981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.866107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.866230 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.866344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.866579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.866694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.866814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.866917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.867036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.867158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.867284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.867399 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.867503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.867616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.867729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.867854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.867969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.868108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.868225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.868338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.868451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.868576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.868691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.868805 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.868918 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.869031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.869144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.869266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.869381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.869516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.869631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.869744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.869857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.869983 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.870098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.870218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.870333 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.870447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.870566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.871217 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.871346 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.871461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.871578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.871692 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.871805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.871927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.872048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.872219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.872341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.872463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.872585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.872777 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.872900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.873015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.873128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.873241 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.873354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.873476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.873599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.873726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.873840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.873954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.874067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.874180 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.874296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.874417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.874533 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.874646 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.874759 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.874880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.874994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.875119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.875233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.875346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.875459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.875573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.875686 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.875808 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.875922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.876035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.876166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.876281 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.876406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.876522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] auth.external = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.876636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] auth.external = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.876756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.876870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.876983 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.877127 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.877252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.877375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.877491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] auth.password = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.877606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] auth.password = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.877720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] auth.token = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.877833 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] auth.token = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.877947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.878069 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.878193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.878309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.878423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.878423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.878643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.878643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.878875 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.878991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.879106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.879106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.879319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.879441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] credential.caching = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.879556 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.879556 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] credential.caching = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.879771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.879884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.879998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.879998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.880274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.880389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.880503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.880617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.880732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.880854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.880976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.881091 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.881205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.881319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.881433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.881555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.881671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.881785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.881912 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.881912 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.882128 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.882250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.882363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.882477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.882591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.882716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.882833 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.882956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.883072 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.883186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] federation.caching = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.883300 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.883414 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.883528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] federation.caching = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.883653 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.883769 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.883882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.883997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.884123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.884243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.884370 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.884488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.884602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.884715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.884829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.884953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.885077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.885192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.885307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.885428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.885549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.885670 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.885793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.885909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.886024 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.caching = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.886138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.886303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.886954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.caching = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.887145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.887262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.887378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.887493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.887607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.887734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.887851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.887965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.888110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.888229 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.888354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.888482 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.888599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.888713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.888827 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.888941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.889056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.889179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.889295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.889410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.889524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.889638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.889753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.889905 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.890049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.890187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.890320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.890436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.890560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.890685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.890801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.890915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.891030 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.891171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.891302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.891427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.891543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.891657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.891771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.891885 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.892006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.892151 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.892269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.892384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.892498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.892612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.892726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.892849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.892965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.893080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.893194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.893308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.893424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.893546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.893663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.893778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.893892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.894006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.894120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.894243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.894364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.894479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.894594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.894708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.894829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.894952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.895068 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.895182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.895296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.895417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.895534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.895657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.895772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.895887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.896001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.896129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.896245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.896371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.896489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.896604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.896718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.896832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.896946 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.897070 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.897185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.897185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.897648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.897648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.897864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.897987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.898102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.898217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.898331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.898331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.898544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.898670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.898785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.898900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.899014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.899129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.899243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.899243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.899467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.899467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.899730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.899845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.899967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.900131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.900248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.900363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.900476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.900591 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.900718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.900835 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.900949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.901063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.901177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.901309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.901431 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.901554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.901669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.901785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.901900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.902014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.902136 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.902252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.902366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.902481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.902595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.902710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.902834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.902950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.903064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.903179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.903292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.903407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.903528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.903644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.903758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.903863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.903976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.904107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.904235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.904356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.904471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.904584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.904704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.904820 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.904943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.905059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.905173 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.905287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.905401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.905515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.905636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.905752 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.905866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.905981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.906095 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.906208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.906330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.906453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.906568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.906682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.906795 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.906908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.907032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.907148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.907261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.907375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.907489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.907602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] resource.caching = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.907724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.907839 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.907953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.908118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.908237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] resource.caching = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.908379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.908505 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.908622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.908737 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.908851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.908964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.909086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.909209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.909325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.909439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.909553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.909667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.909781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.909968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.910085 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] role.caching = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.910200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.910314 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] role.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.910427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.910541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.910663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] role.caching = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.910778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.910892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] role.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.911006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.911128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.911245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.911372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.911489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.911603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.911718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.911839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.911966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.912134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.912254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.912368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.912483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.912596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.912723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.912839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.912954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.913067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.913181 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.913296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.913419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.913535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.913648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.913762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.913762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.913977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.913977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.914201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.914320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.914435 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.914549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.914662 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.914788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.914911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.915027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.915141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.915256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.915371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.915494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.915609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.915723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.915837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.915952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.916082 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.916211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.916328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.916444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.916564 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.916679 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.916793 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.916916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.917031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.917258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.917372 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.917495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.917611 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.917724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.917838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.917951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.918065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.918187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.918303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] token.caching = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.918417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.918532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.918646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.918760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.918881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] token.caching = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.918997 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.919110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.919225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.919339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.919452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.919575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.919691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.919804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.919918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.920032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.920220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.920349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.920466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.920580 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.920695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.920808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.920922 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.920922 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.921145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.921145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.921358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.921472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.921585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.921709 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.921824 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.921937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.922058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.922173 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.922287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.922409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.922524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.connection = **** {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.922646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.922771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.922886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.923000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.923147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.connection = **** {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.923277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.923425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.923570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.923570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.923825 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.923953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.924089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.924207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.924358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.924477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.924593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.924722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.924851 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.924966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.925089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.925206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.925320 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.925435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.925549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.925664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.925787 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.925905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.926020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.926135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.926250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.926365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.926488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.926605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.926719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.926833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.926949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.927063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 17 07:05:41.927186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.927186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.927186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 07:05:41.927503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80118) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 17 07:05:41.956754 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:41.956932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:41.957144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:41.957230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:41.957396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:41.957627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:41.965844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:41.966014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:41.993531 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:41.993753 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:41.993838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:41.993991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.047126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.047904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.095562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.095800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.096482 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.096724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.161967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.162193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.162281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.162422 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.162523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.162645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.162728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.162864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.162946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.163085 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.163176 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.163341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.186382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.186618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.186849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.186946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.187072 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.187155 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.187290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.187372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.187504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.187590 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.187717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.187799 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.187965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.188208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.198691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.199590 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.211767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.211957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.212277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.212427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.212511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.212664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.212746 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.212891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.212980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.213111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.213192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.213331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.213552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.213771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80118) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 07:05:42.230713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.230926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 07:05:42.231196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.231279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.231407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 07:05:42.231500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 07:05:42.231716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.231797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.231927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 07:05:42.232009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.232228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.232313 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.232486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.232567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.232746 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.233008 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.243255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.243542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.243811 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.243893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.244113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.244210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.244404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.244689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.252409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.253041 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.255694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.255975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.256292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.256375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.256578 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.256660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.256862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.256964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.257123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.257206 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.257391 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.257481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.257663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.257745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.257933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.258013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.258206 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.258294 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.258470 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.258550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.258743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.259013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.286245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.286881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.287858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.288166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.288534 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.288817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.293451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.293837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.294118 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.294201 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.294518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.294808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.301421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.301705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.302135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.302422 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.306199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.306488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.306771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.306927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.307053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.307219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.307329 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.307508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.307590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.307796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.307876 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.308091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.308194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.308405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.308679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.308954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.327379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.327656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.328322 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.328609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.332280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.333190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.335922 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.336346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.336907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.337349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.341905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.342971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.346440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.346698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.350099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.350382 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.350466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.350674 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.350755 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.350957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.351045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.351234 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.351314 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.351512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.351593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.351792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.371932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.372236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.372363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.372509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.372653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.372776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.372931 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.373196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.380684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.381172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.384745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.384745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.384947 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.384947 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.385899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.386182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.386939 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.387494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.387563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.387625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.387696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.387760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.387821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.388037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.401124 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.401407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.401689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.401782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.402506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.402609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.402791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.402873 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.403079 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.403162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.403412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.403502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.403733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.404017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.421408 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.421986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.425607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.427303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.428979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.430734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.432429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.432714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.433008 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.434502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.436346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.438010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.438540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.438646 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.440378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.442013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.445306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.445602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.445873 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.445998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.446281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.447503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.451289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.451583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.451977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.452295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.456000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.456315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.456599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.461436 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.461727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.462055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.465052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80118) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.465338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.465623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.465903 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.466179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.466572 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.466854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.467136 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.467417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.467690 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.467965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.468269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.468538 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.468801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80118) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.470432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 07:05:42.470710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.470992 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.471273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.471546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.471924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.472232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.472513 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.472794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.473069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.473345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.473624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.473894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.474157 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 07:05:42.506503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fdac4289c0 pid: 80118 (default app) Jun 17 07:05:42.509940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cbf04-90a9-4c2c-acf1-e6b84e90e1e2 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:42.510239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cbf04-90a9-4c2c-acf1-e6b84e90e1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:42.510524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cbf04-90a9-4c2c-acf1-e6b84e90e1e2 None None] PATH_INFO: `/v3` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:42.512274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fdac4289c0 pid: 80117 (default app) Jun 17 07:05:42.515303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Fri Jun 17 07:05:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:05:50.188197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458e7ea-fd9c-4d36-b277-29d6bdeb597a None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:50.188756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458e7ea-fd9c-4d36-b277-29d6bdeb597a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:50.188756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458e7ea-fd9c-4d36-b277-29d6bdeb597a None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:50.193063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:05:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:05:50.198220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a6169-4a1e-4ed8-a88a-7878a3ed6c9d None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:50.198636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a6169-4a1e-4ed8-a88a-7878a3ed6c9d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:50.198786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a6169-4a1e-4ed8-a88a-7878a3ed6c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:50.268599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: INFO dbcounter [None req-642a6169-4a1e-4ed8-a88a-7878a3ed6c9d None None] Registered counter for database keystone Jun 17 07:05:50.290071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG oslo_db.sqlalchemy.engines [None req-642a6169-4a1e-4ed8-a88a-7878a3ed6c9d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80118) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 17 07:05:50.296747 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writer thread running {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 17 07:05:50.474590 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG passlib.handlers.bcrypt [None req-642a6169-4a1e-4ed8-a88a-7878a3ed6c9d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80118) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 17 07:05:50.474957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG passlib.handlers.bcrypt [None req-642a6169-4a1e-4ed8-a88a-7878a3ed6c9d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80118) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 17 07:05:50.530256 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-642a6169-4a1e-4ed8-a88a-7878a3ed6c9d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:05:50.531887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-642a6169-4a1e-4ed8-a88a-7878a3ed6c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:05:50.655429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:05:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 17 07:05:50.818053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458e7ea-fd9c-4d36-b277-29d6bdeb597a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:50.888191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: INFO dbcounter [None req-d458e7ea-fd9c-4d36-b277-29d6bdeb597a None None] Registered counter for database keystone Jun 17 07:05:50.906941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG oslo_db.sqlalchemy.engines [None req-d458e7ea-fd9c-4d36-b277-29d6bdeb597a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80117) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 17 07:05:50.913012 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writer thread running {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 17 07:05:51.029860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458e7ea-fd9c-4d36-b277-29d6bdeb597a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:51.034078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60463e23-77cd-4387-b23a-96aeaa2b890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:51.034732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60463e23-77cd-4387-b23a-96aeaa2b890f admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:51.035017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60463e23-77cd-4387-b23a-96aeaa2b890f admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:51.035292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60463e23-77cd-4387-b23a-96aeaa2b890f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:51.043192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60463e23-77cd-4387-b23a-96aeaa2b890f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:51.094219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: WARNING keystone.common.rbac_enforcer.enforcer [None req-60463e23-77cd-4387-b23a-96aeaa2b890f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 17 07:05:51.094509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60463e23-77cd-4387-b23a-96aeaa2b890f admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:51.103748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Fri Jun 17 07:05:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 07:05:51.109689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a6169-4a1e-4ed8-a88a-7878a3ed6c9d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:51.124767 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a6169-4a1e-4ed8-a88a-7878a3ed6c9d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:51.129236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b206f-8ea5-427e-8b1d-4c48263cdff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:51.130381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b206f-8ea5-427e-8b1d-4c48263cdff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:51.130678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b206f-8ea5-427e-8b1d-4c48263cdff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:51.130961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b206f-8ea5-427e-8b1d-4c48263cdff3 admin admin] PATH_INFO: `/v3/projects/7107699553c04a40be061877b81300e3` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:51.132533 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b206f-8ea5-427e-8b1d-4c48263cdff3 admin admin] RBAC: Authorizing `identity:get_project(project_id=7107699553c04a40be061877b81300e3)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:51.183580 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: WARNING keystone.common.rbac_enforcer.enforcer [None req-144b206f-8ea5-427e-8b1d-4c48263cdff3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 17 07:05:51.183928 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b206f-8ea5-427e-8b1d-4c48263cdff3 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:51.188613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Fri Jun 17 07:05:51 2022] GET /identity/v3/projects/7107699553c04a40be061877b81300e3 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:05:53.252907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c182da-f4da-467e-887c-35a9e939ebd0 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:53.253439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c182da-f4da-467e-887c-35a9e939ebd0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:53.253533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c182da-f4da-467e-887c-35a9e939ebd0 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:53.257588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:05:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:05:53.263416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da8832-05b1-4c87-aa2a-fa1fc9f807ef None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:53.263831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da8832-05b1-4c87-aa2a-fa1fc9f807ef None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:53.263986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da8832-05b1-4c87-aa2a-fa1fc9f807ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:53.282641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-03da8832-05b1-4c87-aa2a-fa1fc9f807ef None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:05:53.284272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-03da8832-05b1-4c87-aa2a-fa1fc9f807ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:05:53.292333 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:05:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 17 07:05:53.314026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c182da-f4da-467e-887c-35a9e939ebd0 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:53.326428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c182da-f4da-467e-887c-35a9e939ebd0 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:53.330578 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff769bb-1bda-42d0-9c2c-2f99694173d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:53.331204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff769bb-1bda-42d0-9c2c-2f99694173d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:53.331492 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff769bb-1bda-42d0-9c2c-2f99694173d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:53.331771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff769bb-1bda-42d0-9c2c-2f99694173d0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:53.339369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff769bb-1bda-42d0-9c2c-2f99694173d0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:53.341206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff769bb-1bda-42d0-9c2c-2f99694173d0 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:53.349807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Fri Jun 17 07:05:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:05:53.354702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da8832-05b1-4c87-aa2a-fa1fc9f807ef None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:53.363465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da8832-05b1-4c87-aa2a-fa1fc9f807ef None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:53.367773 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98966a0d-7e26-482d-85c2-fcd14f71af9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:53.368660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98966a0d-7e26-482d-85c2-fcd14f71af9e admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:53.368945 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98966a0d-7e26-482d-85c2-fcd14f71af9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:53.369222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98966a0d-7e26-482d-85c2-fcd14f71af9e admin admin] PATH_INFO: `/v3/users/0bf6e418289c4b9c81b349d812dd7d25` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:53.370592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98966a0d-7e26-482d-85c2-fcd14f71af9e admin admin] RBAC: Authorizing `identity:get_user(user_id=0bf6e418289c4b9c81b349d812dd7d25)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:53.373024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98966a0d-7e26-482d-85c2-fcd14f71af9e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:53.374562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Fri Jun 17 07:05:53 2022] GET /identity/v3/users/0bf6e418289c4b9c81b349d812dd7d25 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:05:55.535182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43561d1a-bb21-4fab-a4ae-e3f57322e76a None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:55.535709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43561d1a-bb21-4fab-a4ae-e3f57322e76a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:55.535709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43561d1a-bb21-4fab-a4ae-e3f57322e76a None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:55.537108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:05:55.542194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73054fbd-2e45-4eed-85eb-4741c7787237 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:55.542588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73054fbd-2e45-4eed-85eb-4741c7787237 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:55.542921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73054fbd-2e45-4eed-85eb-4741c7787237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:55.551529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a374d-1c2a-4712-b5fd-c56236333921 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:55.551818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a374d-1c2a-4712-b5fd-c56236333921 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:55.552130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a374d-1c2a-4712-b5fd-c56236333921 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:55.553423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:05:55.558229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99eac1-f1e5-4c70-a118-9da3a0ec6835 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:55.559046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99eac1-f1e5-4c70-a118-9da3a0ec6835 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:55.559429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99eac1-f1e5-4c70-a118-9da3a0ec6835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:55.561689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-73054fbd-2e45-4eed-85eb-4741c7787237 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:05:55.562879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-73054fbd-2e45-4eed-85eb-4741c7787237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:05:55.599005 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG passlib.handlers.bcrypt [None req-3b99eac1-f1e5-4c70-a118-9da3a0ec6835 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80117) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 17 07:05:55.599421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG passlib.handlers.bcrypt [None req-3b99eac1-f1e5-4c70-a118-9da3a0ec6835 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80117) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 17 07:05:55.641697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:05:55.645330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3b99eac1-f1e5-4c70-a118-9da3a0ec6835 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:05:55.646944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3b99eac1-f1e5-4c70-a118-9da3a0ec6835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:05:55.664989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73054fbd-2e45-4eed-85eb-4741c7787237 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:55.688555 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:05:55.745558 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:05:55.767196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73054fbd-2e45-4eed-85eb-4741c7787237 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:55.768334 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99eac1-f1e5-4c70-a118-9da3a0ec6835 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:55.800352 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd269dd-5c90-485e-a63f-f3a1ed1e6ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:55.801142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd269dd-5c90-485e-a63f-f3a1ed1e6ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:55.801433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd269dd-5c90-485e-a63f-f3a1ed1e6ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:55.801707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd269dd-5c90-485e-a63f-f3a1ed1e6ae8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:55.812648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd269dd-5c90-485e-a63f-f3a1ed1e6ae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-afd269dd-5c90-485e-a63f-f3a1ed1e6ae8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:05:55.823990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:05:55.825593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:05:55.826432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:05:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:05:55.831379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd269dd-5c90-485e-a63f-f3a1ed1e6ae8 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:55.866277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99eac1-f1e5-4c70-a118-9da3a0ec6835 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:55.868561 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:05:55.903780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde60ef-b59c-405b-af83-ccfa37621bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:55.904656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde60ef-b59c-405b-af83-ccfa37621bfe None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:55.904948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde60ef-b59c-405b-af83-ccfa37621bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:55.905232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde60ef-b59c-405b-af83-ccfa37621bfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:55.911987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:05:55.930058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd269dd-5c90-485e-a63f-f3a1ed1e6ae8 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:55.963209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccc68a-8938-46fe-9f0f-d55a157d8942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:55.963872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccc68a-8938-46fe-9f0f-d55a157d8942 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:55.964199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccc68a-8938-46fe-9f0f-d55a157d8942 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:55.964483 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccc68a-8938-46fe-9f0f-d55a157d8942 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:55.965771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccc68a-8938-46fe-9f0f-d55a157d8942 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:05:55.966081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccc68a-8938-46fe-9f0f-d55a157d8942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:55.967479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccc68a-8938-46fe-9f0f-d55a157d8942 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:55.971869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde60ef-b59c-405b-af83-ccfa37621bfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:55.973396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde60ef-b59c-405b-af83-ccfa37621bfe None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:55.978859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:05:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:05:55.984264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccc68a-8938-46fe-9f0f-d55a157d8942 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:56.071086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 07:05:56.077290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde60ef-b59c-405b-af83-ccfa37621bfe None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:56.079877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:05:56.081340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccc68a-8938-46fe-9f0f-d55a157d8942 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:56.114676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25af1bf-30cd-4055-b5e7-0a83e6438abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:56.115543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25af1bf-30cd-4055-b5e7-0a83e6438abb None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:56.115830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25af1bf-30cd-4055-b5e7-0a83e6438abb None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:56.116138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25af1bf-30cd-4055-b5e7-0a83e6438abb None admin] PATH_INFO: `/v3/users/0bf6e418289c4b9c81b349d812dd7d25` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:56.117339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25af1bf-30cd-4055-b5e7-0a83e6438abb None admin] RBAC: Authorizing `identity:get_user(user_id=0bf6e418289c4b9c81b349d812dd7d25)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:56.118802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25af1bf-30cd-4055-b5e7-0a83e6438abb None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:56.120471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:05:55 2022] GET /identity/v3/users/0bf6e418289c4b9c81b349d812dd7d25 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:05:56.126213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25af1bf-30cd-4055-b5e7-0a83e6438abb None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:56.140308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:05:56.177695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde60ef-b59c-405b-af83-ccfa37621bfe None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:56.209293 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c46c69-a171-4e52-ba58-a5e051e9fab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:56.210082 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c46c69-a171-4e52-ba58-a5e051e9fab5 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:56.210365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c46c69-a171-4e52-ba58-a5e051e9fab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:56.210640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c46c69-a171-4e52-ba58-a5e051e9fab5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:56.224049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25af1bf-30cd-4055-b5e7-0a83e6438abb None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:56.229059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c46c69-a171-4e52-ba58-a5e051e9fab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:56.231143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c46c69-a171-4e52-ba58-a5e051e9fab5 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-77c46c69-a171-4e52-ba58-a5e051e9fab5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:05:56.251612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:05:56.252623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:05:56.253706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:05:56.253706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:05:56.253706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:05:56.253706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:05:56.253706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:05:56.258420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c46c69-a171-4e52-ba58-a5e051e9fab5 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:56.259960 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e6134-2e36-4ec3-bc07-76bf8b79e48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:56.260792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e6134-2e36-4ec3-bc07-76bf8b79e48c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:56.261249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e6134-2e36-4ec3-bc07-76bf8b79e48c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:56.261526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e6134-2e36-4ec3-bc07-76bf8b79e48c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:56.263509 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e6134-2e36-4ec3-bc07-76bf8b79e48c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:56.264982 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e6134-2e36-4ec3-bc07-76bf8b79e48c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:56.266573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:05:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:05:56.271989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e6134-2e36-4ec3-bc07-76bf8b79e48c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:56.314965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:05:56.349715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:05:56.359970 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c46c69-a171-4e52-ba58-a5e051e9fab5 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:56.369445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e6134-2e36-4ec3-bc07-76bf8b79e48c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:56.398718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982624de-e2ba-475b-96a4-fe00223fd048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:56.399388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982624de-e2ba-475b-96a4-fe00223fd048 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:56.399680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982624de-e2ba-475b-96a4-fe00223fd048 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:56.399958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982624de-e2ba-475b-96a4-fe00223fd048 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:56.400721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982624de-e2ba-475b-96a4-fe00223fd048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:05:56.401030 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982624de-e2ba-475b-96a4-fe00223fd048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:56.402941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982624de-e2ba-475b-96a4-fe00223fd048 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:56.403138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40554147-a07d-4f3a-ab4e-c34610584d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:56.403798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40554147-a07d-4f3a-ab4e-c34610584d2f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:56.404107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40554147-a07d-4f3a-ab4e-c34610584d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:56.404394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40554147-a07d-4f3a-ab4e-c34610584d2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:56.405232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40554147-a07d-4f3a-ab4e-c34610584d2f None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.domain.id=default, user.id=0bf6e418289c4b9c81b349d812dd7d25) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:05:56.405547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40554147-a07d-4f3a-ab4e-c34610584d2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:56.406995 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40554147-a07d-4f3a-ab4e-c34610584d2f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:56.413737 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 11/20] 10.222.0.33 () {56 vars in 1322 bytes} [Fri Jun 17 07:05:56 2022] GET /identity/v3/role_assignments?user.id=0bf6e418289c4b9c81b349d812dd7d25&scope.domain.id=default&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:05:56.421413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 10/21] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:05:58.935022 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f36687-5434-410e-a1e4-5525ff5ce5dc None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:58.936118 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f36687-5434-410e-a1e4-5525ff5ce5dc None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:58.936627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f36687-5434-410e-a1e4-5525ff5ce5dc None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:58.938302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 12/22] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:05:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:05:58.946381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab0b59-bbca-4fbc-b99e-51cecfa7067d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:58.947335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab0b59-bbca-4fbc-b99e-51cecfa7067d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:58.947836 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab0b59-bbca-4fbc-b99e-51cecfa7067d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:58.977137 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c4ab0b59-bbca-4fbc-b99e-51cecfa7067d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:05:58.978694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c4ab0b59-bbca-4fbc-b99e-51cecfa7067d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:05:58.984460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298438d8-7076-48a3-9b6f-72f745694c66 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:58.985097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298438d8-7076-48a3-9b6f-72f745694c66 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:58.985571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298438d8-7076-48a3-9b6f-72f745694c66 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:58.987141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 13/23] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:05:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:05:59.001285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978036e2-ec32-4f2d-87e9-753e5ba3f8ff None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:59.001900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978036e2-ec32-4f2d-87e9-753e5ba3f8ff None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:59.002395 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978036e2-ec32-4f2d-87e9-753e5ba3f8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:59.041079 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-978036e2-ec32-4f2d-87e9-753e5ba3f8ff None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:05:59.042679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-978036e2-ec32-4f2d-87e9-753e5ba3f8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:05:59.096881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 11/24] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:05:59.129655 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d654d6a-2a91-4a92-881f-6cfbf0a8e3a1 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:59.130348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d654d6a-2a91-4a92-881f-6cfbf0a8e3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:59.130824 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d654d6a-2a91-4a92-881f-6cfbf0a8e3a1 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:59.132407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 12/25] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:05:59.137796 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d654d6a-2a91-4a92-881f-6cfbf0a8e3a1 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:59.190729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 14/26] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:05:59.192023 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d5322-9315-4563-9846-01f73307f01d None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:59.192341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d5322-9315-4563-9846-01f73307f01d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:59.192627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d5322-9315-4563-9846-01f73307f01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:59.216428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:05:59.227345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-fb7d5322-9315-4563-9846-01f73307f01d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:05:59.228597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-fb7d5322-9315-4563-9846-01f73307f01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:05:59.327130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d654d6a-2a91-4a92-881f-6cfbf0a8e3a1 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:59.350588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 15/27] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:05:59.351206 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d5322-9315-4563-9846-01f73307f01d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:59.357844 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:05:59.408897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea4417-a28c-4b47-8910-872f89337634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:59.410245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea4417-a28c-4b47-8910-872f89337634 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:59.411669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea4417-a28c-4b47-8910-872f89337634 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:59.412326 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea4417-a28c-4b47-8910-872f89337634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:59.429829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea4417-a28c-4b47-8910-872f89337634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:59.431729 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea4417-a28c-4b47-8910-872f89337634 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-03ea4417-a28c-4b47-8910-872f89337634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:05:59.468988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:05:59.469822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:05:59.470609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:05:59.470609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:05:59.470609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:05:59.470609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:05:59.473476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 13/28] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:05:59.474584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea4417-a28c-4b47-8910-872f89337634 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:59.554491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d5322-9315-4563-9846-01f73307f01d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:59.619102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c97dd-1b57-456d-9f1e-f0e667390c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:59.620476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c97dd-1b57-456d-9f1e-f0e667390c3a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:59.622344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c97dd-1b57-456d-9f1e-f0e667390c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:59.622979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c97dd-1b57-456d-9f1e-f0e667390c3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:59.642231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c97dd-1b57-456d-9f1e-f0e667390c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-a84c97dd-1b57-456d-9f1e-f0e667390c3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:05:59.652821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:05:59.653698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:05:59.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:05:59.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:05:59.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:05:59.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:05:59.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:05:59.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:05:59.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:05:59.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:05:59.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:05:59.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:05:59.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:05:59.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:05:59.656795 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 16/29] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:05:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:05:59.657745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c97dd-1b57-456d-9f1e-f0e667390c3a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:59.662676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:05:59.666793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea4417-a28c-4b47-8910-872f89337634 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:59.709232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6901ac5-91f9-4406-be22-972388c35966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:59.710279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6901ac5-91f9-4406-be22-972388c35966 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:59.710782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6901ac5-91f9-4406-be22-972388c35966 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:59.711476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6901ac5-91f9-4406-be22-972388c35966 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:59.713817 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6901ac5-91f9-4406-be22-972388c35966 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:59.715534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6901ac5-91f9-4406-be22-972388c35966 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:59.801209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:05:59.841328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c97dd-1b57-456d-9f1e-f0e667390c3a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:05:59.860032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 14/30] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jun 17 07:05:59 2022] POST /identity/v3/roles => generated 219 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:05:59.861191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6901ac5-91f9-4406-be22-972388c35966 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:05:59.877751 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0e4c8-1623-4aa3-b371-24bd0f7cc3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:05:59.878668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0e4c8-1623-4aa3-b371-24bd0f7cc3eb None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:59.879099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0e4c8-1623-4aa3-b371-24bd0f7cc3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:59.879537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0e4c8-1623-4aa3-b371-24bd0f7cc3eb None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:59.880455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0e4c8-1623-4aa3-b371-24bd0f7cc3eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:05:59.880922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0e4c8-1623-4aa3-b371-24bd0f7cc3eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:05:59.882476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0e4c8-1623-4aa3-b371-24bd0f7cc3eb None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:05:59.919047 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 17/31] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:05:59.921968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e027a8-2f06-46c9-867c-e88605c9201f None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:05:59.924133 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e027a8-2f06-46c9-867c-e88605c9201f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:05:59.925481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e027a8-2f06-46c9-867c-e88605c9201f None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:05:59.928317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 18/32] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:05:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:05:59.930779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e027a8-2f06-46c9-867c-e88605c9201f None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:00.066074 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6901ac5-91f9-4406-be22-972388c35966 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:00.104519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e027a8-2f06-46c9-867c-e88605c9201f None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:00.116598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdf7c4-ea18-4fb0-a709-b72864c6e7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:00.118006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdf7c4-ea18-4fb0-a709-b72864c6e7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:00.118371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdf7c4-ea18-4fb0-a709-b72864c6e7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:00.118665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdf7c4-ea18-4fb0-a709-b72864c6e7f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:00.119437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdf7c4-ea18-4fb0-a709-b72864c6e7f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:00.119750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdf7c4-ea18-4fb0-a709-b72864c6e7f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:00.121960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdf7c4-ea18-4fb0-a709-b72864c6e7f5 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:00.139236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:00.142775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 15/33] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:05:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:00.145202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fdbcf-fc65-43c5-8f1c-4283d1fbc0e4 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:00.145726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fdbcf-fc65-43c5-8f1c-4283d1fbc0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:00.146229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fdbcf-fc65-43c5-8f1c-4283d1fbc0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:00.160157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7a721-635d-436c-84cc-49d8bca94b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:00.161131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7a721-635d-436c-84cc-49d8bca94b28 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:00.161595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7a721-635d-436c-84cc-49d8bca94b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:00.162100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7a721-635d-436c-84cc-49d8bca94b28 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:00.163045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7a721-635d-436c-84cc-49d8bca94b28 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:00.164758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7a721-635d-436c-84cc-49d8bca94b28 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:00.291261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-b07fdbcf-fc65-43c5-8f1c-4283d1fbc0e4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:00.293322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b07fdbcf-fc65-43c5-8f1c-4283d1fbc0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:00.312600 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:00.378664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 16/34] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:00.379949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fdbcf-fc65-43c5-8f1c-4283d1fbc0e4 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:00.430311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 19/35] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:05:59 2022] POST /identity/v3/projects => generated 301 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:00.431848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfdc48-a296-4ad3-825a-160282058fbe None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:00.432171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfdc48-a296-4ad3-825a-160282058fbe None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:00.432502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfdc48-a296-4ad3-825a-160282058fbe None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:00.433819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 20/36] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:00.434816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfdc48-a296-4ad3-825a-160282058fbe None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:00.513470 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fdbcf-fc65-43c5-8f1c-4283d1fbc0e4 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:00.553445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3bd8e0-6686-43e6-8a02-92a50595936a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:00.554348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bd8e0-6686-43e6-8a02-92a50595936a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:00.554641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bd8e0-6686-43e6-8a02-92a50595936a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:00.554928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bd8e0-6686-43e6-8a02-92a50595936a None admin] PATH_INFO: `/v3/users/0bf6e418289c4b9c81b349d812dd7d25` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:00.556671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3bd8e0-6686-43e6-8a02-92a50595936a None admin] RBAC: Authorizing `identity:get_user(user_id=0bf6e418289c4b9c81b349d812dd7d25)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:00.558193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3bd8e0-6686-43e6-8a02-92a50595936a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:00.559160 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfdc48-a296-4ad3-825a-160282058fbe None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:00.560000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 17/37] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:00 2022] GET /identity/v3/users/0bf6e418289c4b9c81b349d812dd7d25 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:00.561636 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1766e-7c76-4dad-a3d2-76dff58377cc None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:00.561935 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1766e-7c76-4dad-a3d2-76dff58377cc None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:00.562222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1766e-7c76-4dad-a3d2-76dff58377cc None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:00.567714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 18/38] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:00.567714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed305cf-ccfc-42d1-82e4-15f87dab3d4e None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:00.567714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed305cf-ccfc-42d1-82e4-15f87dab3d4e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:00.567714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed305cf-ccfc-42d1-82e4-15f87dab3d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:00.584416 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-fed305cf-ccfc-42d1-82e4-15f87dab3d4e None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:00.585708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-fed305cf-ccfc-42d1-82e4-15f87dab3d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:00.602609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:00.607129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0123d6-4505-4048-8985-5ee5f7e54b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:00.607812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0123d6-4505-4048-8985-5ee5f7e54b5f None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:00.608125 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0123d6-4505-4048-8985-5ee5f7e54b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:00.608413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0123d6-4505-4048-8985-5ee5f7e54b5f None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:00.609331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0123d6-4505-4048-8985-5ee5f7e54b5f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:00.610855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0123d6-4505-4048-8985-5ee5f7e54b5f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:00.628143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 21/39] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jun 17 07:06:00 2022] POST /identity/v3/roles => generated 225 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:00.629167 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0123d6-4505-4048-8985-5ee5f7e54b5f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:00.676179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 19/40] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:00.679512 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684965d9-427e-4438-971a-11d4bdee1273 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:00.679512 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684965d9-427e-4438-971a-11d4bdee1273 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:00.679512 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684965d9-427e-4438-971a-11d4bdee1273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:00.681187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:00.699111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-684965d9-427e-4438-971a-11d4bdee1273 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:00.700590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-684965d9-427e-4438-971a-11d4bdee1273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:00.758059 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0123d6-4505-4048-8985-5ee5f7e54b5f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:00.781140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 20/41] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:00.784288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684965d9-427e-4438-971a-11d4bdee1273 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:00.805172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264dcbe-e138-40df-91e2-f0cf57793f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:00.806089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264dcbe-e138-40df-91e2-f0cf57793f5b None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:00.806371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264dcbe-e138-40df-91e2-f0cf57793f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:00.806645 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264dcbe-e138-40df-91e2-f0cf57793f5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:00.807946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264dcbe-e138-40df-91e2-f0cf57793f5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:00.809584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264dcbe-e138-40df-91e2-f0cf57793f5b None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:00.811559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 22/42] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:00.812417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264dcbe-e138-40df-91e2-f0cf57793f5b None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:00.878178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:00.899720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684965d9-427e-4438-971a-11d4bdee1273 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:00.922322 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264dcbe-e138-40df-91e2-f0cf57793f5b None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:00.924801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:00.934359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821b9cf-f236-4203-a824-1333258dba1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:00.935154 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821b9cf-f236-4203-a824-1333258dba1c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:00.935447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821b9cf-f236-4203-a824-1333258dba1c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:00.935727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821b9cf-f236-4203-a824-1333258dba1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:00.936942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821b9cf-f236-4203-a824-1333258dba1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:00.938402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821b9cf-f236-4203-a824-1333258dba1c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:00.940053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 21/43] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:00.940811 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821b9cf-f236-4203-a824-1333258dba1c None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:00.962941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2439e8-9aed-417d-88b6-a046f0de178a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:00.963619 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2439e8-9aed-417d-88b6-a046f0de178a None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:00.963911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2439e8-9aed-417d-88b6-a046f0de178a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:00.964232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2439e8-9aed-417d-88b6-a046f0de178a None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:00.965562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2439e8-9aed-417d-88b6-a046f0de178a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:00.966968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2439e8-9aed-417d-88b6-a046f0de178a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:00.979519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 23/44] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jun 17 07:06:00 2022] POST /identity/v3/roles => generated 223 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:00.980332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2439e8-9aed-417d-88b6-a046f0de178a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:01.060914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821b9cf-f236-4203-a824-1333258dba1c None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:01.094793 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2439e8-9aed-417d-88b6-a046f0de178a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:01.098173 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3e603-0706-4521-ae03-ecb067dec11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:01.099083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3e603-0706-4521-ae03-ecb067dec11d None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:01.099372 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3e603-0706-4521-ae03-ecb067dec11d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:01.099654 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3e603-0706-4521-ae03-ecb067dec11d None admin] PATH_INFO: `/v3/domains/default/users/0bf6e418289c4b9c81b349d812dd7d25/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:01.101811 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3e603-0706-4521-ae03-ecb067dec11d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0bf6e418289c4b9c81b349d812dd7d25, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:01.103311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3e603-0706-4521-ae03-ecb067dec11d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:01.119771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 22/45] 10.222.0.33 () {58 vars in 1361 bytes} [Fri Jun 17 07:06:00 2022] PUT /identity/v3/domains/default/users/0bf6e418289c4b9c81b349d812dd7d25/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:01.147651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1349e-7d31-4991-9a70-d401dbc6c764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:01.149387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1349e-7d31-4991-9a70-d401dbc6c764 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:01.149681 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1349e-7d31-4991-9a70-d401dbc6c764 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:01.150111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1349e-7d31-4991-9a70-d401dbc6c764 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:01.150902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1349e-7d31-4991-9a70-d401dbc6c764 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:01.153188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1349e-7d31-4991-9a70-d401dbc6c764 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:01.187299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 24/46] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:06:00 2022] POST /identity/v3/projects => generated 312 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:03.253712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c07d-7a22-44ee-9c55-1614e822a0c7 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:03.254247 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c07d-7a22-44ee-9c55-1614e822a0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:03.254329 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c07d-7a22-44ee-9c55-1614e822a0c7 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:03.255677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 23/47] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:03.261531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471530a2-e78a-46b1-becd-40cfdb5b1fbd None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:03.262537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471530a2-e78a-46b1-becd-40cfdb5b1fbd None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:03.262623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471530a2-e78a-46b1-becd-40cfdb5b1fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:03.279963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-471530a2-e78a-46b1-becd-40cfdb5b1fbd None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:03.281515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-471530a2-e78a-46b1-becd-40cfdb5b1fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:03.291934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:03.297725 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a0554-c717-42f7-9d59-2856291d990d None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:03.298041 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a0554-c717-42f7-9d59-2856291d990d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:03.298326 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a0554-c717-42f7-9d59-2856291d990d None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:03.299679 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 24/48] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:06:03.305022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fdf58-803b-4756-a3ff-4e2e05fe99cf None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:03.305317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fdf58-803b-4756-a3ff-4e2e05fe99cf None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:03.305600 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fdf58-803b-4756-a3ff-4e2e05fe99cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:03.316949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:03.322723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-bf8fdf58-803b-4756-a3ff-4e2e05fe99cf None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:03.324985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-bf8fdf58-803b-4756-a3ff-4e2e05fe99cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:03.352648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 25/49] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:03.375183 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471530a2-e78a-46b1-becd-40cfdb5b1fbd None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:03.391199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 25/50] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:03.414511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fdf58-803b-4756-a3ff-4e2e05fe99cf None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:03.474287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471530a2-e78a-46b1-becd-40cfdb5b1fbd None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:03.511079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fdf58-803b-4756-a3ff-4e2e05fe99cf None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:03.511981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4f972-29ff-460a-a2c2-d3808f293675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:03.512808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4f972-29ff-460a-a2c2-d3808f293675 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:03.513094 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4f972-29ff-460a-a2c2-d3808f293675 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:03.513377 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4f972-29ff-460a-a2c2-d3808f293675 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:03.524010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4f972-29ff-460a-a2c2-d3808f293675 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-29d4f972-29ff-460a-a2c2-d3808f293675 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:03.532291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:03.533323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:03.534278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 26/51] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:06:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:06:03.539282 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4f972-29ff-460a-a2c2-d3808f293675 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:03.546707 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17d53a-3f9b-41df-be37-23652e3f5b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:03.547506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17d53a-3f9b-41df-be37-23652e3f5b9e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:03.547798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17d53a-3f9b-41df-be37-23652e3f5b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:03.548097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17d53a-3f9b-41df-be37-23652e3f5b9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:03.548575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:03.549479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc17d53a-3f9b-41df-be37-23652e3f5b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:03.551400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc17d53a-3f9b-41df-be37-23652e3f5b9e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:03.553063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 26/52] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:03.558458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17d53a-3f9b-41df-be37-23652e3f5b9e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:03.563061 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:03.637864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4f972-29ff-460a-a2c2-d3808f293675 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:03.657287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17d53a-3f9b-41df-be37-23652e3f5b9e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:03.671816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020384b-1543-4fb5-9f84-ad9461d6dcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:03.672526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020384b-1543-4fb5-9f84-ad9461d6dcd0 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:03.672819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020384b-1543-4fb5-9f84-ad9461d6dcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:03.673101 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020384b-1543-4fb5-9f84-ad9461d6dcd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:03.673859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020384b-1543-4fb5-9f84-ad9461d6dcd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:03.674174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020384b-1543-4fb5-9f84-ad9461d6dcd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:03.675570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020384b-1543-4fb5-9f84-ad9461d6dcd0 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:03.683489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 27/53] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:03.688891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020384b-1543-4fb5-9f84-ad9461d6dcd0 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:03.690527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8469c7-7e85-404c-bc68-6bb16fa07af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:03.691193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8469c7-7e85-404c-bc68-6bb16fa07af9 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:03.691481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8469c7-7e85-404c-bc68-6bb16fa07af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:03.691760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8469c7-7e85-404c-bc68-6bb16fa07af9 None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:03.692492 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8469c7-7e85-404c-bc68-6bb16fa07af9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:03.693930 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8469c7-7e85-404c-bc68-6bb16fa07af9 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:03.722169 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 27/54] 10.222.0.33 () {60 vars in 1140 bytes} [Fri Jun 17 07:06:03 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:06:03.757570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:03.796163 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020384b-1543-4fb5-9f84-ad9461d6dcd0 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:03.832379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba67fa-7175-41ac-8d14-ac1c6bb40e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:03.833273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba67fa-7175-41ac-8d14-ac1c6bb40e25 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:03.833558 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba67fa-7175-41ac-8d14-ac1c6bb40e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:03.833835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba67fa-7175-41ac-8d14-ac1c6bb40e25 None admin] PATH_INFO: `/v3/users/0bf6e418289c4b9c81b349d812dd7d25` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:03.835103 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba67fa-7175-41ac-8d14-ac1c6bb40e25 None admin] RBAC: Authorizing `identity:get_user(user_id=0bf6e418289c4b9c81b349d812dd7d25)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:03.836628 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba67fa-7175-41ac-8d14-ac1c6bb40e25 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:03.838320 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 28/55] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:03 2022] GET /identity/v3/users/0bf6e418289c4b9c81b349d812dd7d25 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:03.843986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8469c7-7e85-404c-bc68-6bb16fa07af9 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:03.901016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:03.945600 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8469c7-7e85-404c-bc68-6bb16fa07af9 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:03.979408 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77545b-077e-4b6d-8c1f-310655efd083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:03.980229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77545b-077e-4b6d-8c1f-310655efd083 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:03.980520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77545b-077e-4b6d-8c1f-310655efd083 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:03.981338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77545b-077e-4b6d-8c1f-310655efd083 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:03.982500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77545b-077e-4b6d-8c1f-310655efd083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:03.983935 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77545b-077e-4b6d-8c1f-310655efd083 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:03.985577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 28/56] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:03.991199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba67fa-7175-41ac-8d14-ac1c6bb40e25 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:04.089577 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba67fa-7175-41ac-8d14-ac1c6bb40e25 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:04.124028 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1267bbb-4334-41fb-9b7f-e4b6ef8ae044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:04.124703 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1267bbb-4334-41fb-9b7f-e4b6ef8ae044 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:04.124990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1267bbb-4334-41fb-9b7f-e4b6ef8ae044 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:04.125271 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1267bbb-4334-41fb-9b7f-e4b6ef8ae044 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:04.126103 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1267bbb-4334-41fb-9b7f-e4b6ef8ae044 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.domain.id=default, user.id=0bf6e418289c4b9c81b349d812dd7d25) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:04.126420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1267bbb-4334-41fb-9b7f-e4b6ef8ae044 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:04.127866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1267bbb-4334-41fb-9b7f-e4b6ef8ae044 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:04.130743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:04.134559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 29/57] 10.222.0.33 () {56 vars in 1322 bytes} [Fri Jun 17 07:06:03 2022] GET /identity/v3/role_assignments?user.id=0bf6e418289c4b9c81b349d812dd7d25&scope.domain.id=default&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:05.820446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9d673-e6eb-4ce2-9b8f-ba52ecfb236d None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:05.820965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9d673-e6eb-4ce2-9b8f-ba52ecfb236d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:05.821048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9d673-e6eb-4ce2-9b8f-ba52ecfb236d None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:05.822355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 29/58] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:05.827302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39ec8-9eb1-4a99-b12f-379f7feefe44 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:05.827669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39ec8-9eb1-4a99-b12f-379f7feefe44 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:05.827875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39ec8-9eb1-4a99-b12f-379f7feefe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:05.844563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-68f39ec8-9eb1-4a99-b12f-379f7feefe44 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:05.845746 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-68f39ec8-9eb1-4a99-b12f-379f7feefe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:05.908687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 30/59] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:05.932198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9d673-e6eb-4ce2-9b8f-ba52ecfb236d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:06.027187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9d673-e6eb-4ce2-9b8f-ba52ecfb236d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:06.049896 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:06.060155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3bca0-1c74-4541-be94-490ff01af936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:06.060936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3bca0-1c74-4541-be94-490ff01af936 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:06.061221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3bca0-1c74-4541-be94-490ff01af936 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:06.061520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3bca0-1c74-4541-be94-490ff01af936 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:06.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3bca0-1c74-4541-be94-490ff01af936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:06.064180 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3bca0-1c74-4541-be94-490ff01af936 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:06.065756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 30/60] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:06.071083 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39ec8-9eb1-4a99-b12f-379f7feefe44 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:06.164563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39ec8-9eb1-4a99-b12f-379f7feefe44 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:06.191511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:06.197398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745424f8-eb3f-411f-8932-66aff48c059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:06.198055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745424f8-eb3f-411f-8932-66aff48c059e None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:06.198344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745424f8-eb3f-411f-8932-66aff48c059e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:06.198621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745424f8-eb3f-411f-8932-66aff48c059e None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:06.199317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745424f8-eb3f-411f-8932-66aff48c059e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:06.201309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745424f8-eb3f-411f-8932-66aff48c059e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:06.225642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 31/61] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:06:06 2022] POST /identity/v3/users => generated 275 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:09.368908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ec3c9-72fa-48dd-8b12-c539d6edb69a None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:09.369459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ec3c9-72fa-48dd-8b12-c539d6edb69a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:09.369459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ec3c9-72fa-48dd-8b12-c539d6edb69a None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:09.371082 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 31/62] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:09.376465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e668b9a-6bf1-41ff-92a0-11f401b3a242 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:09.376852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e668b9a-6bf1-41ff-92a0-11f401b3a242 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:09.377028 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e668b9a-6bf1-41ff-92a0-11f401b3a242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:09.404279 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-8e668b9a-6bf1-41ff-92a0-11f401b3a242 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:09.405489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8e668b9a-6bf1-41ff-92a0-11f401b3a242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:09.489290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 32/63] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:09.525634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ec3c9-72fa-48dd-8b12-c539d6edb69a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:09.662834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ec3c9-72fa-48dd-8b12-c539d6edb69a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:09.717789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb97e00-8838-4363-8a8e-e9c9db1010bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:09.718615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb97e00-8838-4363-8a8e-e9c9db1010bb None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:09.718903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb97e00-8838-4363-8a8e-e9c9db1010bb None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:09.719183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb97e00-8838-4363-8a8e-e9c9db1010bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:09.737912 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb97e00-8838-4363-8a8e-e9c9db1010bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-dbb97e00-8838-4363-8a8e-e9c9db1010bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:09.747652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:09.748602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:09.749359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:09.749359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:09.749359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:09.749359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:09.749359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:09.749359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:09.749359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:09.749359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:09.749359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:09.749359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:09.749359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:09.749359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:09.757447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e668b9a-6bf1-41ff-92a0-11f401b3a242 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:09.769397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 32/64] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:09.804534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17174814-f386-4bc5-a727-d34a4e8aaf7f None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:09.804848 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17174814-f386-4bc5-a727-d34a4e8aaf7f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:09.805102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17174814-f386-4bc5-a727-d34a4e8aaf7f None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:09.806603 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 33/65] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:06:09.812544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecdc26-a825-403a-bf5a-c263bf05409c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:09.813094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecdc26-a825-403a-bf5a-c263bf05409c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:09.813583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecdc26-a825-403a-bf5a-c263bf05409c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:09.840867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ffecdc26-a825-403a-bf5a-c263bf05409c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:09.842920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ffecdc26-a825-403a-bf5a-c263bf05409c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:09.929562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:09.933137 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e668b9a-6bf1-41ff-92a0-11f401b3a242 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:09.958660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 34/66] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:09.960153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:09.961715 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2ad98-1252-4794-9ed2-a07b77cfb56b None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:09.962175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2ad98-1252-4794-9ed2-a07b77cfb56b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:09.962612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2ad98-1252-4794-9ed2-a07b77cfb56b None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:09.964484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 35/67] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:09.970244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51952357-9014-4717-b8f0-09b86fea2948 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:09.970788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51952357-9014-4717-b8f0-09b86fea2948 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:09.971286 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51952357-9014-4717-b8f0-09b86fea2948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:09.990536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-51952357-9014-4717-b8f0-09b86fea2948 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:09.992002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-51952357-9014-4717-b8f0-09b86fea2948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:10.003675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdedb2b-9a00-4893-bd21-b0b455f0df0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:10.004438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdedb2b-9a00-4893-bd21-b0b455f0df0c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.005141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdedb2b-9a00-4893-bd21-b0b455f0df0c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.005429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdedb2b-9a00-4893-bd21-b0b455f0df0c None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.006508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdedb2b-9a00-4893-bd21-b0b455f0df0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:10.006930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdedb2b-9a00-4893-bd21-b0b455f0df0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:10.008260 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdedb2b-9a00-4893-bd21-b0b455f0df0c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:10.017025 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 33/68] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:10.017777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdedb2b-9a00-4893-bd21-b0b455f0df0c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:10.065286 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 36/69] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:10.066445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99125788-4c1c-4d6f-85ff-fbe140f36b0d None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.066739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99125788-4c1c-4d6f-85ff-fbe140f36b0d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.067020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99125788-4c1c-4d6f-85ff-fbe140f36b0d None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.068301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 37/70] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:10.069143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99125788-4c1c-4d6f-85ff-fbe140f36b0d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:10.136037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdedb2b-9a00-4893-bd21-b0b455f0df0c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:10.174075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35c738-5164-417e-9543-9c1983534c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:10.174920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35c738-5164-417e-9543-9c1983534c9d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.175214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35c738-5164-417e-9543-9c1983534c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.175498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35c738-5164-417e-9543-9c1983534c9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.185320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99125788-4c1c-4d6f-85ff-fbe140f36b0d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:10.189661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35c738-5164-417e-9543-9c1983534c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-8e35c738-5164-417e-9543-9c1983534c9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:10.197990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:10.198902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 34/71] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:06:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:10.204882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dfe2f-789e-4479-a773-1ebee1c1652a None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.205718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dfe2f-789e-4479-a773-1ebee1c1652a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.205718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dfe2f-789e-4479-a773-1ebee1c1652a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.208326 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:10.221086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-e26dfe2f-789e-4479-a773-1ebee1c1652a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:10.222275 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e26dfe2f-789e-4479-a773-1ebee1c1652a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:10.232023 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5ef63-db13-4bba-8952-b13bf0376633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:10.233013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5ef63-db13-4bba-8952-b13bf0376633 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.233304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5ef63-db13-4bba-8952-b13bf0376633 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.233584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5ef63-db13-4bba-8952-b13bf0376633 None admin] PATH_INFO: `/v3/users/8fb531e37b334e7ea25ec28ef9d7d181` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.249359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5ef63-db13-4bba-8952-b13bf0376633 None admin] RBAC: Authorizing `identity:get_user(user_id=8fb531e37b334e7ea25ec28ef9d7d181)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:10.250834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5ef63-db13-4bba-8952-b13bf0376633 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:10.253075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 38/72] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:10 2022] GET /identity/v3/users/8fb531e37b334e7ea25ec28ef9d7d181 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:10.253263 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:10.253953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5ef63-db13-4bba-8952-b13bf0376633 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:10.291423 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 35/73] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:10.292306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dfe2f-789e-4479-a773-1ebee1c1652a None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:10.370772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5ef63-db13-4bba-8952-b13bf0376633 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:10.392111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dfe2f-789e-4479-a773-1ebee1c1652a None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:10.407567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5db43a-7fed-472b-bf58-e526b5521792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:10.408417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5db43a-7fed-472b-bf58-e526b5521792 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.408714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5db43a-7fed-472b-bf58-e526b5521792 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.409001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5db43a-7fed-472b-bf58-e526b5521792 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.419605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5db43a-7fed-472b-bf58-e526b5521792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:10.426157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8b9a5-f30d-47ce-b0c9-66beb76bb493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:10.426821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8b9a5-f30d-47ce-b0c9-66beb76bb493 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.427109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8b9a5-f30d-47ce-b0c9-66beb76bb493 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.427390 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8b9a5-f30d-47ce-b0c9-66beb76bb493 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-4e5db43a-7fed-472b-bf58-e526b5521792 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:10.427490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:10.428487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:10.429276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:10.429276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:10.429276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:10.429276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:10.429276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:10.429276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:10.429276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:10.429276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:10.429276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:10.429276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:10.429276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:10.429276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:10.429950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8b9a5-f30d-47ce-b0c9-66beb76bb493 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:10.429950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8b9a5-f30d-47ce-b0c9-66beb76bb493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:10.430065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 39/74] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:10.430065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e40eb-b898-4aa2-8cbb-024cae714cc8 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.430207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e40eb-b898-4aa2-8cbb-024cae714cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.430279 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8b9a5-f30d-47ce-b0c9-66beb76bb493 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:10.430393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e40eb-b898-4aa2-8cbb-024cae714cc8 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.431796 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 40/75] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:10.436329 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e40eb-b898-4aa2-8cbb-024cae714cc8 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:10.438134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 36/76] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:06:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:10.438796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8b9a5-f30d-47ce-b0c9-66beb76bb493 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:10.467639 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:10.471401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:10.539093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e40eb-b898-4aa2-8cbb-024cae714cc8 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:10.540019 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8b9a5-f30d-47ce-b0c9-66beb76bb493 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:10.572046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238296f8-1fee-48e0-9a2c-37d5a127a614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:10.572715 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238296f8-1fee-48e0-9a2c-37d5a127a614 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.572830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00706632-4389-43d9-bd07-a62b7f520478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:10.573003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238296f8-1fee-48e0-9a2c-37d5a127a614 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.573282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238296f8-1fee-48e0-9a2c-37d5a127a614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.573574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00706632-4389-43d9-bd07-a62b7f520478 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.573863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00706632-4389-43d9-bd07-a62b7f520478 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.574145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00706632-4389-43d9-bd07-a62b7f520478 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.584146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00706632-4389-43d9-bd07-a62b7f520478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-00706632-4389-43d9-bd07-a62b7f520478 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:10.592301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:10.593315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:10.594100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 37/77] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:06:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:10.594893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00706632-4389-43d9-bd07-a62b7f520478 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:10.640109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238296f8-1fee-48e0-9a2c-37d5a127a614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:10.641585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238296f8-1fee-48e0-9a2c-37d5a127a614 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:10.671225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:10.684423 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:10.694117 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00706632-4389-43d9-bd07-a62b7f520478 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:10.727923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43fc416-d0d8-4712-9c43-93e7b28ef7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:10.728627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fc416-d0d8-4712-9c43-93e7b28ef7eb None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.728921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fc416-d0d8-4712-9c43-93e7b28ef7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.729206 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fc416-d0d8-4712-9c43-93e7b28ef7eb None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.729954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43fc416-d0d8-4712-9c43-93e7b28ef7eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:10.730270 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43fc416-d0d8-4712-9c43-93e7b28ef7eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:10.731664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43fc416-d0d8-4712-9c43-93e7b28ef7eb None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:10.739440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 38/78] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:10.745381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 41/79] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:10.745381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238296f8-1fee-48e0-9a2c-37d5a127a614 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:10.745589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b4684-8270-423a-8076-4148d0983bf8 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.745589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b4684-8270-423a-8076-4148d0983bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.745589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b4684-8270-423a-8076-4148d0983bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.761613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c53b4684-8270-423a-8076-4148d0983bf8 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:10.763410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c53b4684-8270-423a-8076-4148d0983bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:10.829861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 39/80] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:10.830701 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b4684-8270-423a-8076-4148d0983bf8 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:10.854729 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238296f8-1fee-48e0-9a2c-37d5a127a614 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:10.883401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:10.890055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d29a4-24ea-4e46-8640-e9a383cf8c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:10.890931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d29a4-24ea-4e46-8640-e9a383cf8c19 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.891226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d29a4-24ea-4e46-8640-e9a383cf8c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.891507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d29a4-24ea-4e46-8640-e9a383cf8c19 None admin] PATH_INFO: `/v3/users/8fb531e37b334e7ea25ec28ef9d7d181` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.892713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d29a4-24ea-4e46-8640-e9a383cf8c19 None admin] RBAC: Authorizing `identity:get_user(user_id=8fb531e37b334e7ea25ec28ef9d7d181)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:10.894185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d29a4-24ea-4e46-8640-e9a383cf8c19 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:10.895833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 42/81] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:10 2022] GET /identity/v3/users/8fb531e37b334e7ea25ec28ef9d7d181 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:10.899116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d29a4-24ea-4e46-8640-e9a383cf8c19 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:10.911403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:10.929857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b4684-8270-423a-8076-4148d0983bf8 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:10.962844 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cc4e5-2cd4-4b19-bc54-415bb3540fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:10.963453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cc4e5-2cd4-4b19-bc54-415bb3540fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:10.963743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cc4e5-2cd4-4b19-bc54-415bb3540fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:10.964027 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cc4e5-2cd4-4b19-bc54-415bb3540fa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:10.964804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cc4e5-2cd4-4b19-bc54-415bb3540fa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:10.965184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cc4e5-2cd4-4b19-bc54-415bb3540fa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:10.967054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cc4e5-2cd4-4b19-bc54-415bb3540fa5 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:10.974997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 40/82] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:10.975778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cc4e5-2cd4-4b19-bc54-415bb3540fa5 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:11.000859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d29a4-24ea-4e46-8640-e9a383cf8c19 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:11.034193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a7857-9b7c-4f36-899c-b7d617a87e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:11.035057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a7857-9b7c-4f36-899c-b7d617a87e0b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:11.035345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a7857-9b7c-4f36-899c-b7d617a87e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:11.035623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a7857-9b7c-4f36-899c-b7d617a87e0b None admin] PATH_INFO: `/v3/users/8fb531e37b334e7ea25ec28ef9d7d181` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:11.036843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a7857-9b7c-4f36-899c-b7d617a87e0b None admin] RBAC: Authorizing `identity:get_user(user_id=8fb531e37b334e7ea25ec28ef9d7d181)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:11.038302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a7857-9b7c-4f36-899c-b7d617a87e0b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:11.039927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 43/83] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:10 2022] GET /identity/v3/users/8fb531e37b334e7ea25ec28ef9d7d181 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:11.040768 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a7857-9b7c-4f36-899c-b7d617a87e0b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:11.073538 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cc4e5-2cd4-4b19-bc54-415bb3540fa5 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:11.104520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:11.106692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18114cbd-2ca4-4ce1-ae07-7ed897dc3d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:11.107622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18114cbd-2ca4-4ce1-ae07-7ed897dc3d39 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:11.107913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18114cbd-2ca4-4ce1-ae07-7ed897dc3d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:11.108293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18114cbd-2ca4-4ce1-ae07-7ed897dc3d39 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:11.109494 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18114cbd-2ca4-4ce1-ae07-7ed897dc3d39 None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:11.111301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18114cbd-2ca4-4ce1-ae07-7ed897dc3d39 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:11.113341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 41/84] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:10 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:11.114269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18114cbd-2ca4-4ce1-ae07-7ed897dc3d39 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:11.129506 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:11.144334 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a7857-9b7c-4f36-899c-b7d617a87e0b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:11.177950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f07f7-c962-4644-8c7b-edafa8446aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:11.178739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f07f7-c962-4644-8c7b-edafa8446aca None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:11.179032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f07f7-c962-4644-8c7b-edafa8446aca None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:11.179306 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f07f7-c962-4644-8c7b-edafa8446aca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:11.180495 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f07f7-c962-4644-8c7b-edafa8446aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:11.181942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f07f7-c962-4644-8c7b-edafa8446aca None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:11.183575 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 44/85] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:11.184391 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f07f7-c962-4644-8c7b-edafa8446aca None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:11.212567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18114cbd-2ca4-4ce1-ae07-7ed897dc3d39 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:11.246060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aace7b0-69dd-4491-b2b3-147ef9782928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:11.246907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aace7b0-69dd-4491-b2b3-147ef9782928 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:11.247208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aace7b0-69dd-4491-b2b3-147ef9782928 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:11.247496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aace7b0-69dd-4491-b2b3-147ef9782928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:11.282543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f07f7-c962-4644-8c7b-edafa8446aca None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:11.315770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aace7b0-69dd-4491-b2b3-147ef9782928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:11.316216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3a5e2-41d0-415c-a1b3-27340bd68705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:11.317168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3a5e2-41d0-415c-a1b3-27340bd68705 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:11.317396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aace7b0-69dd-4491-b2b3-147ef9782928 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:11.317521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3a5e2-41d0-415c-a1b3-27340bd68705 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:11.317807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3a5e2-41d0-415c-a1b3-27340bd68705 None admin] PATH_INFO: `/v3/users/0bf6e418289c4b9c81b349d812dd7d25` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:11.319010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3a5e2-41d0-415c-a1b3-27340bd68705 None admin] RBAC: Authorizing `identity:get_user(user_id=0bf6e418289c4b9c81b349d812dd7d25)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:11.320490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3a5e2-41d0-415c-a1b3-27340bd68705 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:11.322095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 45/86] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:11 2022] GET /identity/v3/users/0bf6e418289c4b9c81b349d812dd7d25 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:11.322849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3a5e2-41d0-415c-a1b3-27340bd68705 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:11.325372 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:11.331980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:11.415897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 42/87] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:11.416784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aace7b0-69dd-4491-b2b3-147ef9782928 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:11.429806 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3a5e2-41d0-415c-a1b3-27340bd68705 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:11.462613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8d217-1d4e-485c-9f37-29d501f19491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:11.463261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8d217-1d4e-485c-9f37-29d501f19491 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:11.463549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8d217-1d4e-485c-9f37-29d501f19491 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:11.463825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8d217-1d4e-485c-9f37-29d501f19491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:11.513808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aace7b0-69dd-4491-b2b3-147ef9782928 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:11.526833 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:11.530291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb8d217-1d4e-485c-9f37-29d501f19491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:11.531761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb8d217-1d4e-485c-9f37-29d501f19491 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:11.533610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:11.549394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43cdf0-0545-478b-9a5f-6f79ca2ea0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:11.550057 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43cdf0-0545-478b-9a5f-6f79ca2ea0dc None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:11.550351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43cdf0-0545-478b-9a5f-6f79ca2ea0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:11.550637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43cdf0-0545-478b-9a5f-6f79ca2ea0dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:11.551475 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e43cdf0-0545-478b-9a5f-6f79ca2ea0dc None admin] RBAC: Adding query filter params (role.id=ce36ba2b01744465803d0533b2a47592, scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5, user.id=8fb531e37b334e7ea25ec28ef9d7d181) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:11.551800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e43cdf0-0545-478b-9a5f-6f79ca2ea0dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:11.553303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e43cdf0-0545-478b-9a5f-6f79ca2ea0dc None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:11.558939 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 43/88] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:11 2022] GET /identity/v3/role_assignments?user.id=8fb531e37b334e7ea25ec28ef9d7d181&scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5&role.id=ce36ba2b01744465803d0533b2a47592 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:11.559699 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43cdf0-0545-478b-9a5f-6f79ca2ea0dc None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:11.658211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 46/89] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:11.659457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8d217-1d4e-485c-9f37-29d501f19491 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:11.673813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43cdf0-0545-478b-9a5f-6f79ca2ea0dc None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:11.711149 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4768a-7054-459b-885d-b76eaa4bfec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:11.712232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4768a-7054-459b-885d-b76eaa4bfec1 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:11.712668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4768a-7054-459b-885d-b76eaa4bfec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:11.713093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4768a-7054-459b-885d-b76eaa4bfec1 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:11.714213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4768a-7054-459b-885d-b76eaa4bfec1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:11.716559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4768a-7054-459b-885d-b76eaa4bfec1 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:11.739525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 44/90] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:06:11 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:11.745158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4768a-7054-459b-885d-b76eaa4bfec1 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:11.780307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8d217-1d4e-485c-9f37-29d501f19491 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:11.781982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:11.794008 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:11.823492 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51041118-4463-4010-b335-da849917cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:11.824511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51041118-4463-4010-b335-da849917cb3e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:11.825526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51041118-4463-4010-b335-da849917cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:11.826082 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51041118-4463-4010-b335-da849917cb3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:11.896158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4768a-7054-459b-885d-b76eaa4bfec1 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:11.904369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51041118-4463-4010-b335-da849917cb3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:11.906355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51041118-4463-4010-b335-da849917cb3e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:11.942321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fa029-319c-4e3f-b6cf-adbd9c133970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:11.942321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fa029-319c-4e3f-b6cf-adbd9c133970 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:11.942603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fa029-319c-4e3f-b6cf-adbd9c133970 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:11.942673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fa029-319c-4e3f-b6cf-adbd9c133970 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:11.944473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fa029-319c-4e3f-b6cf-adbd9c133970 None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:11.945556 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fa029-319c-4e3f-b6cf-adbd9c133970 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:11.947451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 45/91] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:11 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:11.953485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fa029-319c-4e3f-b6cf-adbd9c133970 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:12.032466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 47/92] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:12.033630 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51041118-4463-4010-b335-da849917cb3e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:12.036939 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:12.084730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:12.086907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fa029-319c-4e3f-b6cf-adbd9c133970 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:12.132699 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d724874-d682-4042-8167-58274860ab43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:12.133807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d724874-d682-4042-8167-58274860ab43 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:12.134239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d724874-d682-4042-8167-58274860ab43 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:12.134655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d724874-d682-4042-8167-58274860ab43 None admin] PATH_INFO: `/v3/projects/954126f281db499f89335d0beef28c3d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:12.136032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d724874-d682-4042-8167-58274860ab43 None admin] RBAC: Authorizing `identity:get_project(project_id=954126f281db499f89335d0beef28c3d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:12.138179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d724874-d682-4042-8167-58274860ab43 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:12.140058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 46/93] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:11 2022] GET /identity/v3/projects/954126f281db499f89335d0beef28c3d => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:12.141655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d724874-d682-4042-8167-58274860ab43 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:12.174309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51041118-4463-4010-b335-da849917cb3e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:12.253356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6c86c-e254-4bc6-90bf-6bfa90b6b389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:12.254042 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6c86c-e254-4bc6-90bf-6bfa90b6b389 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:12.254334 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6c86c-e254-4bc6-90bf-6bfa90b6b389 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:12.254617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6c86c-e254-4bc6-90bf-6bfa90b6b389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:12.255473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6c86c-e254-4bc6-90bf-6bfa90b6b389 None admin] RBAC: Adding query filter params (role.id=22d7b181c7e041329c59c3eb6b492cd7, scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5, user.id=8fb531e37b334e7ea25ec28ef9d7d181) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:12.255794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6c86c-e254-4bc6-90bf-6bfa90b6b389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:12.257313 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6c86c-e254-4bc6-90bf-6bfa90b6b389 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:12.264631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d724874-d682-4042-8167-58274860ab43 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:12.275715 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 48/94] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:12 2022] GET /identity/v3/role_assignments?user.id=8fb531e37b334e7ea25ec28ef9d7d181&scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5&role.id=22d7b181c7e041329c59c3eb6b492cd7 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:12.276598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6c86c-e254-4bc6-90bf-6bfa90b6b389 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:12.318969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6816e7-164b-431a-8183-0c9b32d96ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:12.319845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6816e7-164b-431a-8183-0c9b32d96ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:12.320166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6816e7-164b-431a-8183-0c9b32d96ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:12.320456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6816e7-164b-431a-8183-0c9b32d96ad4 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:12.322288 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6816e7-164b-431a-8183-0c9b32d96ad4 None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:12.323814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6816e7-164b-431a-8183-0c9b32d96ad4 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:12.326518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 47/95] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:12 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:12.332261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6816e7-164b-431a-8183-0c9b32d96ad4 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:12.372508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:12.416640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:12.482018 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6c86c-e254-4bc6-90bf-6bfa90b6b389 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:12.549491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf83afc-f47b-4f2d-9b3a-c837f97845dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:12.550176 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf83afc-f47b-4f2d-9b3a-c837f97845dd None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:12.550495 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf83afc-f47b-4f2d-9b3a-c837f97845dd None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:12.550781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf83afc-f47b-4f2d-9b3a-c837f97845dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:12.551648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf83afc-f47b-4f2d-9b3a-c837f97845dd None admin] RBAC: Adding query filter params (role.id=ce36ba2b01744465803d0533b2a47592, scope.project.id=954126f281db499f89335d0beef28c3d, user.id=8fb531e37b334e7ea25ec28ef9d7d181) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:12.551964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf83afc-f47b-4f2d-9b3a-c837f97845dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:12.553475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf83afc-f47b-4f2d-9b3a-c837f97845dd None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:12.558079 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6816e7-164b-431a-8183-0c9b32d96ad4 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:12.564890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 49/96] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:12 2022] GET /identity/v3/role_assignments?user.id=8fb531e37b334e7ea25ec28ef9d7d181&scope.project.id=954126f281db499f89335d0beef28c3d&role.id=ce36ba2b01744465803d0533b2a47592 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:12.629340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded7d63-0b54-4ae6-b4cf-680a02026c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:12.629933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded7d63-0b54-4ae6-b4cf-680a02026c13 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:12.630227 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded7d63-0b54-4ae6-b4cf-680a02026c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:12.630513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded7d63-0b54-4ae6-b4cf-680a02026c13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:12.631419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded7d63-0b54-4ae6-b4cf-680a02026c13 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5, user.id=0bf6e418289c4b9c81b349d812dd7d25) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:12.631746 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded7d63-0b54-4ae6-b4cf-680a02026c13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:12.633298 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded7d63-0b54-4ae6-b4cf-680a02026c13 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:12.640677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 48/97] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:12 2022] GET /identity/v3/role_assignments?user.id=0bf6e418289c4b9c81b349d812dd7d25&scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:14.779856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa2786-61c0-4132-b254-37613aa104ea None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:14.780395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa2786-61c0-4132-b254-37613aa104ea None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:14.780469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa2786-61c0-4132-b254-37613aa104ea None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:14.781968 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 50/98] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:14.782950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0097136-504b-4861-80d2-05bf055efadc None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:14.783720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0097136-504b-4861-80d2-05bf055efadc None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:14.784267 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0097136-504b-4861-80d2-05bf055efadc None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:14.786081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 49/99] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:14.791415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2edd8-a50c-424f-8b84-37258598fb74 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:14.791710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2edd8-a50c-424f-8b84-37258598fb74 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:14.791995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2edd8-a50c-424f-8b84-37258598fb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:14.792320 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8969e5-cf4c-4ee3-b756-67e74e593fc7 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:14.792811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8969e5-cf4c-4ee3-b756-67e74e593fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:14.793307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8969e5-cf4c-4ee3-b756-67e74e593fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:14.821403 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-45e2edd8-a50c-424f-8b84-37258598fb74 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:14.822861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-45e2edd8-a50c-424f-8b84-37258598fb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:14.829138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6d8969e5-cf4c-4ee3-b756-67e74e593fc7 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:14.830487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6d8969e5-cf4c-4ee3-b756-67e74e593fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:14.937381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 51/100] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:14.973961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2edd8-a50c-424f-8b84-37258598fb74 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:15.010425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 50/101] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:15.041531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8969e5-cf4c-4ee3-b756-67e74e593fc7 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:15.069227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:15.105207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:15.145463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2edd8-a50c-424f-8b84-37258598fb74 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:15.188683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8969e5-cf4c-4ee3-b756-67e74e593fc7 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:15.196577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01f5eb-491a-4a23-a99b-8f70ea1f8642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:15.197773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f5eb-491a-4a23-a99b-8f70ea1f8642 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.198266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f5eb-491a-4a23-a99b-8f70ea1f8642 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.198743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01f5eb-491a-4a23-a99b-8f70ea1f8642 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.212384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01f5eb-491a-4a23-a99b-8f70ea1f8642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-fa01f5eb-491a-4a23-a99b-8f70ea1f8642 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:15.222295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:15.223207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:15.224034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:15.224034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:15.224034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:15.224034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:15.224034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:15.224034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:15.224034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:15.224034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:15.224034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:15.224034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:15.224034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:15.224034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:15.227270 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 52/102] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:15.228925 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8a72e-21e6-4461-bcfb-56e29e20c761 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.229426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8a72e-21e6-4461-bcfb-56e29e20c761 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.229923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8a72e-21e6-4461-bcfb-56e29e20c761 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.231485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 53/103] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:15.240995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8a72e-21e6-4461-bcfb-56e29e20c761 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:15.277290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0bf29-a439-41d9-b439-ac9be21db1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:15.278962 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0bf29-a439-41d9-b439-ac9be21db1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.279589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0bf29-a439-41d9-b439-ac9be21db1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.280821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0bf29-a439-41d9-b439-ac9be21db1c0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.294424 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0bf29-a439-41d9-b439-ac9be21db1c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-bca0bf29-a439-41d9-b439-ac9be21db1c0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:15.305486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:15.306545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:15.307493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:15.307493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:15.307493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:15.307493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:15.307493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:15.307493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:15.307493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:15.307493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:15.307493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:15.307493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:15.307493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:15.307493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:15.309861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 51/104] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:15.311600 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed0784-2b7e-4b34-848a-ee2a5f9f9a5b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.312081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed0784-2b7e-4b34-848a-ee2a5f9f9a5b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.312524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed0784-2b7e-4b34-848a-ee2a5f9f9a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.332825 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-3fed0784-2b7e-4b34-848a-ee2a5f9f9a5b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:15.334691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3fed0784-2b7e-4b34-848a-ee2a5f9f9a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:15.365723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8a72e-21e6-4461-bcfb-56e29e20c761 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:15.406884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ccd972-ac0e-431d-909d-3da43fb54ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:15.407869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ccd972-ac0e-431d-909d-3da43fb54ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.408641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ccd972-ac0e-431d-909d-3da43fb54ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.409122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ccd972-ac0e-431d-909d-3da43fb54ce6 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.410071 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ccd972-ac0e-431d-909d-3da43fb54ce6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:15.410617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ccd972-ac0e-431d-909d-3da43fb54ce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:15.416246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:15.417353 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ccd972-ac0e-431d-909d-3da43fb54ce6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:15.420661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 52/105] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:15.422003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302fd71-1694-40bb-8249-d4b59cb1d0c1 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.422301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302fd71-1694-40bb-8249-d4b59cb1d0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.422590 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302fd71-1694-40bb-8249-d4b59cb1d0c1 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.424013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 53/106] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:15.429923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 54/107] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:15.431043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302fd71-1694-40bb-8249-d4b59cb1d0c1 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:15.434259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8d4cb-2dc2-434e-9d8c-c4d89d6e06b7 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.436187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8d4cb-2dc2-434e-9d8c-c4d89d6e06b7 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.436756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8d4cb-2dc2-434e-9d8c-c4d89d6e06b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.468542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-20c8d4cb-2dc2-434e-9d8c-c4d89d6e06b7 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:15.470220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-20c8d4cb-2dc2-434e-9d8c-c4d89d6e06b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:15.475715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:15.550369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 55/108] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:15.551437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8d4cb-2dc2-434e-9d8c-c4d89d6e06b7 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:15.584157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302fd71-1694-40bb-8249-d4b59cb1d0c1 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:15.620736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b887e2-1ae0-4cea-a359-43c9423f7152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:15.621410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b887e2-1ae0-4cea-a359-43c9423f7152 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.622329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b887e2-1ae0-4cea-a359-43c9423f7152 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.622611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b887e2-1ae0-4cea-a359-43c9423f7152 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.623362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b887e2-1ae0-4cea-a359-43c9423f7152 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:15.623672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b887e2-1ae0-4cea-a359-43c9423f7152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:15.625098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b887e2-1ae0-4cea-a359-43c9423f7152 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:15.638457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 54/109] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:15.638457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b887e2-1ae0-4cea-a359-43c9423f7152 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:15.667351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8d4cb-2dc2-434e-9d8c-c4d89d6e06b7 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:15.682789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:15.709424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee04d07-3197-4c17-a7bb-f270c8d5aadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:15.710303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee04d07-3197-4c17-a7bb-f270c8d5aadd None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.710589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee04d07-3197-4c17-a7bb-f270c8d5aadd None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.710862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee04d07-3197-4c17-a7bb-f270c8d5aadd None admin] PATH_INFO: `/v3/users/8fb531e37b334e7ea25ec28ef9d7d181` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.712985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee04d07-3197-4c17-a7bb-f270c8d5aadd None admin] RBAC: Authorizing `identity:get_user(user_id=8fb531e37b334e7ea25ec28ef9d7d181)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:15.714505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee04d07-3197-4c17-a7bb-f270c8d5aadd None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:15.722480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 56/110] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:15 2022] GET /identity/v3/users/8fb531e37b334e7ea25ec28ef9d7d181 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:15.722480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51e0e0-a887-49d1-9f46-e51b01ce0220 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.722480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51e0e0-a887-49d1-9f46-e51b01ce0220 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.722480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51e0e0-a887-49d1-9f46-e51b01ce0220 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.722480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 57/111] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:15.722480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51e0e0-a887-49d1-9f46-e51b01ce0220 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:15.726434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:15.745570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b887e2-1ae0-4cea-a359-43c9423f7152 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:15.782210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595221a-f5d4-40b5-89c1-12719f6cc63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:15.783186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595221a-f5d4-40b5-89c1-12719f6cc63d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.783714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595221a-f5d4-40b5-89c1-12719f6cc63d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.783714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595221a-f5d4-40b5-89c1-12719f6cc63d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.785073 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595221a-f5d4-40b5-89c1-12719f6cc63d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:15.786663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595221a-f5d4-40b5-89c1-12719f6cc63d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:15.788337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 55/112] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:15.789103 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595221a-f5d4-40b5-89c1-12719f6cc63d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:15.842237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51e0e0-a887-49d1-9f46-e51b01ce0220 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:15.878029 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15299f50-d192-4662-90f2-9620dde9075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:15.878930 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15299f50-d192-4662-90f2-9620dde9075e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.879221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15299f50-d192-4662-90f2-9620dde9075e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.879552 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15299f50-d192-4662-90f2-9620dde9075e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.890653 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15299f50-d192-4662-90f2-9620dde9075e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:15.892986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595221a-f5d4-40b5-89c1-12719f6cc63d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-15299f50-d192-4662-90f2-9620dde9075e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:15.899064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:15.900252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 58/113] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:06:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:15.906686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15299f50-d192-4662-90f2-9620dde9075e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:15.937762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641f86a-249e-4a20-b634-0793ab438de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:15.938779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641f86a-249e-4a20-b634-0793ab438de0 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.939146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641f86a-249e-4a20-b634-0793ab438de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.939509 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641f86a-249e-4a20-b634-0793ab438de0 None admin] PATH_INFO: `/v3/users/7b5dec3a875348d9839b5ccf4cc3b9c5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.947110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:15.955579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641f86a-249e-4a20-b634-0793ab438de0 None admin] RBAC: Authorizing `identity:get_user(user_id=7b5dec3a875348d9839b5ccf4cc3b9c5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:15.957127 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641f86a-249e-4a20-b634-0793ab438de0 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:15.959575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 56/114] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:15 2022] GET /identity/v3/users/7b5dec3a875348d9839b5ccf4cc3b9c5 => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:15.960711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a95832-3f90-4666-bc74-69a601df1657 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:15.961007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a95832-3f90-4666-bc74-69a601df1657 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:15.961290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a95832-3f90-4666-bc74-69a601df1657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:15.978293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-42a95832-3f90-4666-bc74-69a601df1657 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:15.979488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-42a95832-3f90-4666-bc74-69a601df1657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:15.981512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:16.016657 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15299f50-d192-4662-90f2-9620dde9075e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:16.047770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 57/115] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:16.048626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a95832-3f90-4666-bc74-69a601df1657 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:16.054510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201deab-a3db-4d4c-911f-40b26bb93d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:16.055377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201deab-a3db-4d4c-911f-40b26bb93d67 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:16.055666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201deab-a3db-4d4c-911f-40b26bb93d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:16.056374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201deab-a3db-4d4c-911f-40b26bb93d67 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:16.057618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201deab-a3db-4d4c-911f-40b26bb93d67 None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:16.059810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201deab-a3db-4d4c-911f-40b26bb93d67 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:16.061700 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 59/116] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:15 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:16.062459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201deab-a3db-4d4c-911f-40b26bb93d67 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:16.144684 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a95832-3f90-4666-bc74-69a601df1657 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:16.164363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201deab-a3db-4d4c-911f-40b26bb93d67 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:16.173134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:16.179311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578c2fe-2911-47d4-95a8-a2a032ed8199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:16.179981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578c2fe-2911-47d4-95a8-a2a032ed8199 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:16.180302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578c2fe-2911-47d4-95a8-a2a032ed8199 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:16.180584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578c2fe-2911-47d4-95a8-a2a032ed8199 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:16.181286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578c2fe-2911-47d4-95a8-a2a032ed8199 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:16.182999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578c2fe-2911-47d4-95a8-a2a032ed8199 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:16.190892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:16.200363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1da3f1-fb7f-4280-8043-152c1662cd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:16.201086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da3f1-fb7f-4280-8043-152c1662cd0d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:16.201378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da3f1-fb7f-4280-8043-152c1662cd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:16.201683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da3f1-fb7f-4280-8043-152c1662cd0d None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:16.202520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1da3f1-fb7f-4280-8043-152c1662cd0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:16.202879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1da3f1-fb7f-4280-8043-152c1662cd0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:16.204293 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1da3f1-fb7f-4280-8043-152c1662cd0d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:16.208123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 58/117] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:06:16 2022] POST /identity/v3/projects => generated 302 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:16.208935 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578c2fe-2911-47d4-95a8-a2a032ed8199 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:16.222144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 60/118] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:06:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:16.224331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311a25a-3a8c-4768-a7fa-689524224c2b None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:16.224627 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311a25a-3a8c-4768-a7fa-689524224c2b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:16.224973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311a25a-3a8c-4768-a7fa-689524224c2b None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:16.228877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 61/119] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:16.228877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311a25a-3a8c-4768-a7fa-689524224c2b None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:16.333425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578c2fe-2911-47d4-95a8-a2a032ed8199 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:16.346942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311a25a-3a8c-4768-a7fa-689524224c2b None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:16.370090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38b8f5-0e1a-4d20-ab9b-5a531ed7bef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:16.371309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38b8f5-0e1a-4d20-ab9b-5a531ed7bef0 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:16.371309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38b8f5-0e1a-4d20-ab9b-5a531ed7bef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:16.371309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38b8f5-0e1a-4d20-ab9b-5a531ed7bef0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:16.395152 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e657536-44ba-478e-bed1-86236f62143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:16.395706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e657536-44ba-478e-bed1-86236f62143f None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:16.396001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e657536-44ba-478e-bed1-86236f62143f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:16.397195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e657536-44ba-478e-bed1-86236f62143f None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5/users/8fb531e37b334e7ea25ec28ef9d7d181/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:16.399188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e657536-44ba-478e-bed1-86236f62143f None admin] RBAC: Authorizing `identity:create_grant(project_id=f1a4c2e64b4747a29a5b7958456ce1f5, user_id=8fb531e37b334e7ea25ec28ef9d7d181, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:16.401574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e657536-44ba-478e-bed1-86236f62143f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:16.415134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 62/120] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:06:16 2022] PUT /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5/users/8fb531e37b334e7ea25ec28ef9d7d181/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:16.415659 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e657536-44ba-478e-bed1-86236f62143f None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:16.446451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:16.458400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:16.465246 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38b8f5-0e1a-4d20-ab9b-5a531ed7bef0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:16.468318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38b8f5-0e1a-4d20-ab9b-5a531ed7bef0 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:16.557962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e657536-44ba-478e-bed1-86236f62143f None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:16.595141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 59/121] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:16.596375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38b8f5-0e1a-4d20-ab9b-5a531ed7bef0 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:16.602338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72def0e-c60d-4ada-a3a0-96a674e3f62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:16.603598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72def0e-c60d-4ada-a3a0-96a674e3f62e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:16.603598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72def0e-c60d-4ada-a3a0-96a674e3f62e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:16.603766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72def0e-c60d-4ada-a3a0-96a674e3f62e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:16.618398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72def0e-c60d-4ada-a3a0-96a674e3f62e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-d72def0e-c60d-4ada-a3a0-96a674e3f62e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:16.627380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:16.628301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:16.629063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 63/122] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:06:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:16.630375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f2c1f-f50b-442a-806a-fc26f56d9278 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:16.630375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f2c1f-f50b-442a-806a-fc26f56d9278 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:16.630534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f2c1f-f50b-442a-806a-fc26f56d9278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:16.650176 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-361f2c1f-f50b-442a-806a-fc26f56d9278 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:16.651301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-361f2c1f-f50b-442a-806a-fc26f56d9278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:16.723810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:16.734680 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38b8f5-0e1a-4d20-ab9b-5a531ed7bef0 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:16.735562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 64/123] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:16.736693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f2c1f-f50b-442a-806a-fc26f56d9278 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:16.766129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:16.776473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9b3c8-6530-4056-9e2a-3d31cc70022f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:16.777809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9b3c8-6530-4056-9e2a-3d31cc70022f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:16.778841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9b3c8-6530-4056-9e2a-3d31cc70022f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:16.779354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9b3c8-6530-4056-9e2a-3d31cc70022f None admin] PATH_INFO: `/v3/users/8fb531e37b334e7ea25ec28ef9d7d181` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:16.780924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9b3c8-6530-4056-9e2a-3d31cc70022f None admin] RBAC: Authorizing `identity:get_user(user_id=8fb531e37b334e7ea25ec28ef9d7d181)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:16.782649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9b3c8-6530-4056-9e2a-3d31cc70022f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:16.784721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 60/124] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:16 2022] GET /identity/v3/users/8fb531e37b334e7ea25ec28ef9d7d181 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:16.786461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9b3c8-6530-4056-9e2a-3d31cc70022f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:16.857751 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f2c1f-f50b-442a-806a-fc26f56d9278 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:16.894282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a29f36-15cd-4932-926d-5f2b1b6eb1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:16.895183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a29f36-15cd-4932-926d-5f2b1b6eb1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:16.895469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a29f36-15cd-4932-926d-5f2b1b6eb1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:16.895748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a29f36-15cd-4932-926d-5f2b1b6eb1f6 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:16.896898 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9b3c8-6530-4056-9e2a-3d31cc70022f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:16.897056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a29f36-15cd-4932-926d-5f2b1b6eb1f6 None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:16.898410 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a29f36-15cd-4932-926d-5f2b1b6eb1f6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:16.901073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 65/125] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:16 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:16.901369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a29f36-15cd-4932-926d-5f2b1b6eb1f6 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:16.935852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b980b6-99f9-4ac1-b3f0-10423617e1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:16.936962 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b980b6-99f9-4ac1-b3f0-10423617e1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:16.937518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b980b6-99f9-4ac1-b3f0-10423617e1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:16.938096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b980b6-99f9-4ac1-b3f0-10423617e1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:16.939055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b980b6-99f9-4ac1-b3f0-10423617e1d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:16.939573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b980b6-99f9-4ac1-b3f0-10423617e1d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:16.941203 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b980b6-99f9-4ac1-b3f0-10423617e1d8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:16.951400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 61/126] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:16.952909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b980b6-99f9-4ac1-b3f0-10423617e1d8 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:16.980990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:17.002000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:17.016481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a29f36-15cd-4932-926d-5f2b1b6eb1f6 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:17.053037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d2e16-e357-4d98-b058-d10d1133abee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:17.053845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d2e16-e357-4d98-b058-d10d1133abee None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:17.054231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d2e16-e357-4d98-b058-d10d1133abee None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:17.054510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d2e16-e357-4d98-b058-d10d1133abee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:17.068992 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d2e16-e357-4d98-b058-d10d1133abee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:17.072852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b980b6-99f9-4ac1-b3f0-10423617e1d8 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-5c8d2e16-e357-4d98-b058-d10d1133abee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:17.077567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:17.078438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:17.079202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 66/127] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:17.079858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d2e16-e357-4d98-b058-d10d1133abee None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:17.114438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20389f89-57ab-426d-b043-371c668e67c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:17.115938 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20389f89-57ab-426d-b043-371c668e67c8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:17.116652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20389f89-57ab-426d-b043-371c668e67c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:17.117235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20389f89-57ab-426d-b043-371c668e67c8 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:17.118849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20389f89-57ab-426d-b043-371c668e67c8 None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:17.120549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20389f89-57ab-426d-b043-371c668e67c8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:17.123401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 62/128] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:16 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:17.125031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20389f89-57ab-426d-b043-371c668e67c8 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:17.192157 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d2e16-e357-4d98-b058-d10d1133abee None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:17.227055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:17.229157 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b55cc-ba00-440f-abd4-770879674a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:17.230000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b55cc-ba00-440f-abd4-770879674a03 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:17.230520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b55cc-ba00-440f-abd4-770879674a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:17.230994 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b55cc-ba00-440f-abd4-770879674a03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:17.233516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b55cc-ba00-440f-abd4-770879674a03 None admin] RBAC: Adding query filter params (role.id=b3b7dfbee8c34760b5acbb12576c8fdd, scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5, user.id=7b5dec3a875348d9839b5ccf4cc3b9c5) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:17.233995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b55cc-ba00-440f-abd4-770879674a03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:17.235805 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b55cc-ba00-440f-abd4-770879674a03 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:17.241737 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20389f89-57ab-426d-b043-371c668e67c8 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:17.242106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 67/129] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:17 2022] GET /identity/v3/role_assignments?user.id=7b5dec3a875348d9839b5ccf4cc3b9c5&scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5&role.id=b3b7dfbee8c34760b5acbb12576c8fdd => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:17.243050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b55cc-ba00-440f-abd4-770879674a03 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:17.264103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:17.290975 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a822733-3e1d-4514-8414-30f50c1052cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:17.292128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a822733-3e1d-4514-8414-30f50c1052cc None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:17.292426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a822733-3e1d-4514-8414-30f50c1052cc None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:17.292710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a822733-3e1d-4514-8414-30f50c1052cc None admin] PATH_INFO: `/v3/users/0bf6e418289c4b9c81b349d812dd7d25` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:17.297121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a822733-3e1d-4514-8414-30f50c1052cc None admin] RBAC: Authorizing `identity:get_user(user_id=0bf6e418289c4b9c81b349d812dd7d25)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:17.298611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a822733-3e1d-4514-8414-30f50c1052cc None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:17.308733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 63/130] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:17 2022] GET /identity/v3/users/0bf6e418289c4b9c81b349d812dd7d25 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:17.309412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a822733-3e1d-4514-8414-30f50c1052cc None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:17.460844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b55cc-ba00-440f-abd4-770879674a03 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:17.495180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a822733-3e1d-4514-8414-30f50c1052cc None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:17.501727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e7a29-bdb8-4574-adf7-2ffb2cae3a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:17.502827 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e7a29-bdb8-4574-adf7-2ffb2cae3a47 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:17.503316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e7a29-bdb8-4574-adf7-2ffb2cae3a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:17.503791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e7a29-bdb8-4574-adf7-2ffb2cae3a47 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:17.504881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e7a29-bdb8-4574-adf7-2ffb2cae3a47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:17.505416 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e7a29-bdb8-4574-adf7-2ffb2cae3a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:17.507111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e7a29-bdb8-4574-adf7-2ffb2cae3a47 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:17.517133 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 68/131] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:17.518394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e7a29-bdb8-4574-adf7-2ffb2cae3a47 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:17.535338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6eedc-9a6c-4721-bbcd-22d5374e6068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:17.536276 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6eedc-9a6c-4721-bbcd-22d5374e6068 None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:17.536589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6eedc-9a6c-4721-bbcd-22d5374e6068 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:17.536970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6eedc-9a6c-4721-bbcd-22d5374e6068 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5/users/8fb531e37b334e7ea25ec28ef9d7d181/roles/22d7b181c7e041329c59c3eb6b492cd7` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:17.539693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6eedc-9a6c-4721-bbcd-22d5374e6068 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1a4c2e64b4747a29a5b7958456ce1f5, user_id=8fb531e37b334e7ea25ec28ef9d7d181, role_id=22d7b181c7e041329c59c3eb6b492cd7)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:17.541345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6eedc-9a6c-4721-bbcd-22d5374e6068 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:17.567859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 64/132] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:06:17 2022] PUT /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5/users/8fb531e37b334e7ea25ec28ef9d7d181/roles/22d7b181c7e041329c59c3eb6b492cd7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:17.568750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6eedc-9a6c-4721-bbcd-22d5374e6068 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:17.612639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:17.661386 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:17.716166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e7a29-bdb8-4574-adf7-2ffb2cae3a47 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:17.760185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bae75b-c2d7-4e44-86c0-34e8268369db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:17.761073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bae75b-c2d7-4e44-86c0-34e8268369db None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:17.761370 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bae75b-c2d7-4e44-86c0-34e8268369db None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:17.761647 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bae75b-c2d7-4e44-86c0-34e8268369db None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:17.763428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bae75b-c2d7-4e44-86c0-34e8268369db None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:17.765361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bae75b-c2d7-4e44-86c0-34e8268369db None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:17.767732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 69/133] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:17 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:17.773529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bae75b-c2d7-4e44-86c0-34e8268369db None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:17.806660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6eedc-9a6c-4721-bbcd-22d5374e6068 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:17.870999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ff97e-ad38-4fd3-a84d-c603075fd970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:17.873522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ff97e-ad38-4fd3-a84d-c603075fd970 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:17.873818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ff97e-ad38-4fd3-a84d-c603075fd970 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:17.874100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ff97e-ad38-4fd3-a84d-c603075fd970 None admin] PATH_INFO: `/v3/users/8fb531e37b334e7ea25ec28ef9d7d181` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:17.875959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ff97e-ad38-4fd3-a84d-c603075fd970 None admin] RBAC: Authorizing `identity:get_user(user_id=8fb531e37b334e7ea25ec28ef9d7d181)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:17.877504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ff97e-ad38-4fd3-a84d-c603075fd970 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:17.879210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 65/134] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:17 2022] GET /identity/v3/users/8fb531e37b334e7ea25ec28ef9d7d181 => generated 275 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:17.888131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ff97e-ad38-4fd3-a84d-c603075fd970 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:17.952006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bae75b-c2d7-4e44-86c0-34e8268369db None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:18.019536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc1558-4614-4886-83f7-1ae3b78e275e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:18.020413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc1558-4614-4886-83f7-1ae3b78e275e None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:18.020701 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc1558-4614-4886-83f7-1ae3b78e275e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:18.020984 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc1558-4614-4886-83f7-1ae3b78e275e None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5/users/0bf6e418289c4b9c81b349d812dd7d25/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:18.023113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc1558-4614-4886-83f7-1ae3b78e275e None admin] RBAC: Authorizing `identity:create_grant(project_id=f1a4c2e64b4747a29a5b7958456ce1f5, user_id=0bf6e418289c4b9c81b349d812dd7d25, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:18.024601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc1558-4614-4886-83f7-1ae3b78e275e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:18.046393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:18.053155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 70/135] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:06:17 2022] PUT /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5/users/0bf6e418289c4b9c81b349d812dd7d25/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:18.064153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ff97e-ad38-4fd3-a84d-c603075fd970 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:18.102588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d96a2d-50a7-4634-bed5-6152b4fefbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:18.103979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d96a2d-50a7-4634-bed5-6152b4fefbb9 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:18.104333 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d96a2d-50a7-4634-bed5-6152b4fefbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:18.104618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d96a2d-50a7-4634-bed5-6152b4fefbb9 None admin] PATH_INFO: `/v3/projects/954126f281db499f89335d0beef28c3d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:18.105919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d96a2d-50a7-4634-bed5-6152b4fefbb9 None admin] RBAC: Authorizing `identity:get_project(project_id=954126f281db499f89335d0beef28c3d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:18.107516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d96a2d-50a7-4634-bed5-6152b4fefbb9 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:18.109448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 66/136] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:17 2022] GET /identity/v3/projects/954126f281db499f89335d0beef28c3d => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:18.115239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc1558-4614-4886-83f7-1ae3b78e275e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:18.123257 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:18.288730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc1558-4614-4886-83f7-1ae3b78e275e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:18.347617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db62dd-4d65-42c6-bfbd-0e449fa68152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:18.348570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db62dd-4d65-42c6-bfbd-0e449fa68152 None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:18.348859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db62dd-4d65-42c6-bfbd-0e449fa68152 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:18.349138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db62dd-4d65-42c6-bfbd-0e449fa68152 None admin] PATH_INFO: `/v3/projects/954126f281db499f89335d0beef28c3d/users/8fb531e37b334e7ea25ec28ef9d7d181/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:18.351232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db62dd-4d65-42c6-bfbd-0e449fa68152 None admin] RBAC: Authorizing `identity:create_grant(project_id=954126f281db499f89335d0beef28c3d, user_id=8fb531e37b334e7ea25ec28ef9d7d181, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:18.352729 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db62dd-4d65-42c6-bfbd-0e449fa68152 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:18.380380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 71/137] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:06:18 2022] PUT /identity/v3/projects/954126f281db499f89335d0beef28c3d/users/8fb531e37b334e7ea25ec28ef9d7d181/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:19.064325 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943a6a4-3190-4c85-b02a-11ec82566bf1 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:19.064834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943a6a4-3190-4c85-b02a-11ec82566bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:19.064932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943a6a4-3190-4c85-b02a-11ec82566bf1 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:19.066334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 67/138] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:19.071546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec9983-a592-4b28-aebf-1f38d3a59a6a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:19.071941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec9983-a592-4b28-aebf-1f38d3a59a6a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:19.072146 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec9983-a592-4b28-aebf-1f38d3a59a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:19.093121 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c6ec9983-a592-4b28-aebf-1f38d3a59a6a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:19.094376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c6ec9983-a592-4b28-aebf-1f38d3a59a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:19.165683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:19.179888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 72/139] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:19.216823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943a6a4-3190-4c85-b02a-11ec82566bf1 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:19.344852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943a6a4-3190-4c85-b02a-11ec82566bf1 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:19.380794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162114d-6161-43af-94d3-64ce38f04936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:19.381600 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162114d-6161-43af-94d3-64ce38f04936 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:19.381716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623d3dd-272f-4a50-93d7-1e94226584bf None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:19.381909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162114d-6161-43af-94d3-64ce38f04936 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:19.382008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623d3dd-272f-4a50-93d7-1e94226584bf None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:19.382159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162114d-6161-43af-94d3-64ce38f04936 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:19.382246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623d3dd-272f-4a50-93d7-1e94226584bf None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:19.383346 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162114d-6161-43af-94d3-64ce38f04936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:19.383595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 73/140] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:19.384842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162114d-6161-43af-94d3-64ce38f04936 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:19.386460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 68/141] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:19.393047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623d3dd-272f-4a50-93d7-1e94226584bf None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:19.397762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40d26f-a346-46b9-b8fb-004f812ff875 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:19.398063 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40d26f-a346-46b9-b8fb-004f812ff875 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:19.398349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40d26f-a346-46b9-b8fb-004f812ff875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:19.425132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-9e40d26f-a346-46b9-b8fb-004f812ff875 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:19.426371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-9e40d26f-a346-46b9-b8fb-004f812ff875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:19.456432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:19.548965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 69/142] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:19.592278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40d26f-a346-46b9-b8fb-004f812ff875 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:19.601729 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623d3dd-272f-4a50-93d7-1e94226584bf None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:19.672796 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92baa220-0c65-441c-879b-b19f03553321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:19.673492 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baa220-0c65-441c-879b-b19f03553321 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:19.673783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baa220-0c65-441c-879b-b19f03553321 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:19.674060 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baa220-0c65-441c-879b-b19f03553321 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:19.674786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92baa220-0c65-441c-879b-b19f03553321 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:19.676262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92baa220-0c65-441c-879b-b19f03553321 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:19.732228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 74/143] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:06:19 2022] POST /identity/v3/users => generated 283 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:19.759961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40d26f-a346-46b9-b8fb-004f812ff875 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:19.830244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd61dec-f15a-46e6-8bec-54caf1511beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:19.831067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd61dec-f15a-46e6-8bec-54caf1511beb None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:19.831349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd61dec-f15a-46e6-8bec-54caf1511beb None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:19.831631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd61dec-f15a-46e6-8bec-54caf1511beb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:19.857623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd61dec-f15a-46e6-8bec-54caf1511beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-cdd61dec-f15a-46e6-8bec-54caf1511beb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:19.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:19.866601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:19.867380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 70/144] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:06:19.875295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92baa220-0c65-441c-879b-b19f03553321 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:19.950280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:20.015400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92baa220-0c65-441c-879b-b19f03553321 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:20.055526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da743210-c3e0-42ce-9c0a-6fa58a9d4de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:20.056233 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da743210-c3e0-42ce-9c0a-6fa58a9d4de7 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:20.056532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da743210-c3e0-42ce-9c0a-6fa58a9d4de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:20.056823 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da743210-c3e0-42ce-9c0a-6fa58a9d4de7 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:20.057544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da743210-c3e0-42ce-9c0a-6fa58a9d4de7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:20.057853 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da743210-c3e0-42ce-9c0a-6fa58a9d4de7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:20.059299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da743210-c3e0-42ce-9c0a-6fa58a9d4de7 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:20.068173 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 75/145] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:20.073355 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd61dec-f15a-46e6-8bec-54caf1511beb None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:20.104425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:20.232170 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123ef1f-b3ac-45bb-accf-203b22e67c6e None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:20.232673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123ef1f-b3ac-45bb-accf-203b22e67c6e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:20.232747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123ef1f-b3ac-45bb-accf-203b22e67c6e None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:20.234244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 76/146] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:06:20.240544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d2d7e-9a49-42f9-b488-2b63a6c48fc0 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:20.241110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d2d7e-9a49-42f9-b488-2b63a6c48fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:20.241595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d2d7e-9a49-42f9-b488-2b63a6c48fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:20.261707 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3e1d2d7e-9a49-42f9-b488-2b63a6c48fc0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:20.263858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3e1d2d7e-9a49-42f9-b488-2b63a6c48fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:20.322008 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd61dec-f15a-46e6-8bec-54caf1511beb None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:20.404186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcac0b-1f7b-4db1-8f8b-2f7420e18a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:20.405110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcac0b-1f7b-4db1-8f8b-2f7420e18a85 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:20.405372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcac0b-1f7b-4db1-8f8b-2f7420e18a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:20.405650 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcac0b-1f7b-4db1-8f8b-2f7420e18a85 None admin] PATH_INFO: `/v3/users/8fb531e37b334e7ea25ec28ef9d7d181` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:20.406970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcac0b-1f7b-4db1-8f8b-2f7420e18a85 None admin] RBAC: Authorizing `identity:get_user(user_id=8fb531e37b334e7ea25ec28ef9d7d181)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:20.408609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcac0b-1f7b-4db1-8f8b-2f7420e18a85 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:20.410260 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 71/147] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:20 2022] GET /identity/v3/users/8fb531e37b334e7ea25ec28ef9d7d181 => generated 275 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:20.410648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 77/148] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:20.416778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcac0b-1f7b-4db1-8f8b-2f7420e18a85 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:20.443712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d2d7e-9a49-42f9-b488-2b63a6c48fc0 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:20.509279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:20.527297 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:20.560411 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcac0b-1f7b-4db1-8f8b-2f7420e18a85 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:20.596059 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b9538-c792-4085-9ec1-d67a34cd53d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:20.596860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b9538-c792-4085-9ec1-d67a34cd53d1 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:20.597216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b9538-c792-4085-9ec1-d67a34cd53d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:20.597566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b9538-c792-4085-9ec1-d67a34cd53d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:20.606957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d2d7e-9a49-42f9-b488-2b63a6c48fc0 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:20.665837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9e43e-a3aa-4219-bf38-705994cf8efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:20.667011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9e43e-a3aa-4219-bf38-705994cf8efe None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:20.667482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9e43e-a3aa-4219-bf38-705994cf8efe None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:20.667973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9e43e-a3aa-4219-bf38-705994cf8efe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:20.669308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b9538-c792-4085-9ec1-d67a34cd53d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:20.671009 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b9538-c792-4085-9ec1-d67a34cd53d1 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:20.687491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9e43e-a3aa-4219-bf38-705994cf8efe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-70a9e43e-a3aa-4219-bf38-705994cf8efe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:20.696388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:20.697237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:20.697998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:20.697998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:20.697998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:20.697998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:20.697998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:20.697998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:20.697998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:20.697998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:20.697998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:20.697998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:20.697998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:20.697998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:20.705518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 78/149] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:06:20.707077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6f373-9e0f-45df-a5aa-8e63441daa9d None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:20.707578 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6f373-9e0f-45df-a5aa-8e63441daa9d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:20.708085 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6f373-9e0f-45df-a5aa-8e63441daa9d None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:20.709591 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 79/150] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:20.714977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4723d-ad7f-4517-8a07-b65f901c06a8 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:20.715582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4723d-ad7f-4517-8a07-b65f901c06a8 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:20.716080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4723d-ad7f-4517-8a07-b65f901c06a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:20.742019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f3d4723d-ad7f-4517-8a07-b65f901c06a8 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:20.743775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f3d4723d-ad7f-4517-8a07-b65f901c06a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:20.756281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:20.781313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 72/151] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:20.782134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b9538-c792-4085-9ec1-d67a34cd53d1 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:20.867088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 80/152] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:20.868311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4723d-ad7f-4517-8a07-b65f901c06a8 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:20.887953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b9538-c792-4085-9ec1-d67a34cd53d1 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:20.921348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:20.925670 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f883d51-4ec7-49fa-bab5-7ecc8a8eb2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:20.926399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f883d51-4ec7-49fa-bab5-7ecc8a8eb2be None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:20.926886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f883d51-4ec7-49fa-bab5-7ecc8a8eb2be None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:20.927170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f883d51-4ec7-49fa-bab5-7ecc8a8eb2be None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:20.927932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f883d51-4ec7-49fa-bab5-7ecc8a8eb2be None admin] RBAC: Adding query filter params (name=reader) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:20.928274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f883d51-4ec7-49fa-bab5-7ecc8a8eb2be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:20.929848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f883d51-4ec7-49fa-bab5-7ecc8a8eb2be None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:20.939011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 73/153] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:20.939866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f883d51-4ec7-49fa-bab5-7ecc8a8eb2be None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:20.992512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:21.035191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4723d-ad7f-4517-8a07-b65f901c06a8 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:21.049228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f883d51-4ec7-49fa-bab5-7ecc8a8eb2be None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:21.083863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8172c5-49e4-49c6-8e88-d9ba39f42391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:21.084741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8172c5-49e4-49c6-8e88-d9ba39f42391 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:21.085805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8172c5-49e4-49c6-8e88-d9ba39f42391 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:21.085805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8172c5-49e4-49c6-8e88-d9ba39f42391 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:21.097237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8172c5-49e4-49c6-8e88-d9ba39f42391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:21.098152 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91430545-de01-4279-ac6b-200fe8bb39d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:21.099243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91430545-de01-4279-ac6b-200fe8bb39d0 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:21.099705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91430545-de01-4279-ac6b-200fe8bb39d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:21.100183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91430545-de01-4279-ac6b-200fe8bb39d0 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:21.104793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91430545-de01-4279-ac6b-200fe8bb39d0 None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-5c8172c5-49e4-49c6-8e88-d9ba39f42391 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:21.106067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:21.106914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:21.107880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 74/154] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:06:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:21.109009 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8172c5-49e4-49c6-8e88-d9ba39f42391 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:21.109440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91430545-de01-4279-ac6b-200fe8bb39d0 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:21.111413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 81/155] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:20 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:21.114012 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91430545-de01-4279-ac6b-200fe8bb39d0 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:21.214329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8172c5-49e4-49c6-8e88-d9ba39f42391 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:21.216023 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:21.251216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728120a8-f069-443d-accd-7dfb90422d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:21.252140 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728120a8-f069-443d-accd-7dfb90422d7d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:21.252436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728120a8-f069-443d-accd-7dfb90422d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:21.252726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728120a8-f069-443d-accd-7dfb90422d7d None admin] PATH_INFO: `/v3/users/7b5dec3a875348d9839b5ccf4cc3b9c5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:21.253970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728120a8-f069-443d-accd-7dfb90422d7d None admin] RBAC: Authorizing `identity:get_user(user_id=7b5dec3a875348d9839b5ccf4cc3b9c5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:21.255454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728120a8-f069-443d-accd-7dfb90422d7d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:21.257282 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 75/156] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:21 2022] GET /identity/v3/users/7b5dec3a875348d9839b5ccf4cc3b9c5 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:21.257885 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728120a8-f069-443d-accd-7dfb90422d7d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:21.261172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91430545-de01-4279-ac6b-200fe8bb39d0 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:21.269054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:21.339106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaf403-8b98-4e2e-a407-f96a517e5ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:21.339772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaf403-8b98-4e2e-a407-f96a517e5ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:21.340057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaf403-8b98-4e2e-a407-f96a517e5ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:21.340352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaf403-8b98-4e2e-a407-f96a517e5ba9 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:21.341155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feaf403-8b98-4e2e-a407-f96a517e5ba9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:21.341460 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feaf403-8b98-4e2e-a407-f96a517e5ba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:21.342915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feaf403-8b98-4e2e-a407-f96a517e5ba9 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:21.366061 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 82/157] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:06:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:21.366996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaf403-8b98-4e2e-a407-f96a517e5ba9 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:21.369758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728120a8-f069-443d-accd-7dfb90422d7d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:21.407609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654e27c-21b4-4074-a1cb-3bfcca33d17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:21.408318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654e27c-21b4-4074-a1cb-3bfcca33d17d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:21.408612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654e27c-21b4-4074-a1cb-3bfcca33d17d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:21.408895 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654e27c-21b4-4074-a1cb-3bfcca33d17d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:21.410224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654e27c-21b4-4074-a1cb-3bfcca33d17d None admin] RBAC: Adding query filter params (role.id=ce36ba2b01744465803d0533b2a47592, scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5, user.id=8fb531e37b334e7ea25ec28ef9d7d181) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:21.410536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654e27c-21b4-4074-a1cb-3bfcca33d17d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:21.412560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654e27c-21b4-4074-a1cb-3bfcca33d17d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:21.419701 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 76/158] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:21 2022] GET /identity/v3/role_assignments?user.id=8fb531e37b334e7ea25ec28ef9d7d181&scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5&role.id=ce36ba2b01744465803d0533b2a47592 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:21.420545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654e27c-21b4-4074-a1cb-3bfcca33d17d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:21.472610 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:21.539011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654e27c-21b4-4074-a1cb-3bfcca33d17d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:21.581319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82585c0e-6da4-4702-b2d5-0a096eaad9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:21.582243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82585c0e-6da4-4702-b2d5-0a096eaad9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:21.582529 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82585c0e-6da4-4702-b2d5-0a096eaad9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:21.582810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82585c0e-6da4-4702-b2d5-0a096eaad9f9 None admin] PATH_INFO: `/v3/users/8fb531e37b334e7ea25ec28ef9d7d181` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:21.584130 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82585c0e-6da4-4702-b2d5-0a096eaad9f9 None admin] RBAC: Authorizing `identity:get_user(user_id=8fb531e37b334e7ea25ec28ef9d7d181)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:21.585920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82585c0e-6da4-4702-b2d5-0a096eaad9f9 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:21.587916 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 77/159] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:21 2022] GET /identity/v3/users/8fb531e37b334e7ea25ec28ef9d7d181 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:21.593547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82585c0e-6da4-4702-b2d5-0a096eaad9f9 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:21.734584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82585c0e-6da4-4702-b2d5-0a096eaad9f9 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:21.760461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:21.771387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaf403-8b98-4e2e-a407-f96a517e5ba9 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:21.773491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15a471-3f4a-49c1-9d5c-34e824057a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:21.774172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15a471-3f4a-49c1-9d5c-34e824057a9f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:21.774467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15a471-3f4a-49c1-9d5c-34e824057a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:21.774754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15a471-3f4a-49c1-9d5c-34e824057a9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:21.851655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15a471-3f4a-49c1-9d5c-34e824057a9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:21.853430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15a471-3f4a-49c1-9d5c-34e824057a9f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:21.909746 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af312c6-cf8b-4292-9ae2-61f5b52fb12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:21.910632 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af312c6-cf8b-4292-9ae2-61f5b52fb12d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:21.910924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af312c6-cf8b-4292-9ae2-61f5b52fb12d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:21.911206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af312c6-cf8b-4292-9ae2-61f5b52fb12d None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:21.912423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af312c6-cf8b-4292-9ae2-61f5b52fb12d None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:21.913880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af312c6-cf8b-4292-9ae2-61f5b52fb12d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:21.915578 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 83/160] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:21 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 549 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:21.917144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6fbb9-3880-43c2-ab53-678cb8e86880 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:21.917280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6fbb9-3880-43c2-ab53-678cb8e86880 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:21.917497 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6fbb9-3880-43c2-ab53-678cb8e86880 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:21.918795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 84/161] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:21.923745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6fbb9-3880-43c2-ab53-678cb8e86880 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:21.962455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:21.966802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 78/162] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 07:06:21.968775 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a1423-61ea-40a8-b42c-222804985566 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:21.969371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a1423-61ea-40a8-b42c-222804985566 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:21.969891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a1423-61ea-40a8-b42c-222804985566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:21.990492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-ff8a1423-61ea-40a8-b42c-222804985566 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:21.992172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ff8a1423-61ea-40a8-b42c-222804985566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:22.017657 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:22.115601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 79/163] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:22.116502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a1423-61ea-40a8-b42c-222804985566 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:22.116766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6fbb9-3880-43c2-ab53-678cb8e86880 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:22.160107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd55ed-b6da-45b2-bfc5-fedff6d1a27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:22.161717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd55ed-b6da-45b2-bfc5-fedff6d1a27a None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:22.162255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd55ed-b6da-45b2-bfc5-fedff6d1a27a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:22.162900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd55ed-b6da-45b2-bfc5-fedff6d1a27a None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5/users/7b5dec3a875348d9839b5ccf4cc3b9c5/roles/b3b7dfbee8c34760b5acbb12576c8fdd` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:22.165143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd55ed-b6da-45b2-bfc5-fedff6d1a27a None admin] RBAC: Authorizing `identity:create_grant(project_id=f1a4c2e64b4747a29a5b7958456ce1f5, user_id=7b5dec3a875348d9839b5ccf4cc3b9c5, role_id=b3b7dfbee8c34760b5acbb12576c8fdd)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:22.166867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd55ed-b6da-45b2-bfc5-fedff6d1a27a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:22.190409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 85/164] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:06:21 2022] PUT /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5/users/7b5dec3a875348d9839b5ccf4cc3b9c5/roles/b3b7dfbee8c34760b5acbb12576c8fdd => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:22.195478 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c03cea-ccd0-4afa-b288-6f31aa40ac44 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:22.196409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c03cea-ccd0-4afa-b288-6f31aa40ac44 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:22.197932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c03cea-ccd0-4afa-b288-6f31aa40ac44 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:22.202900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 86/165] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:22.204039 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c03cea-ccd0-4afa-b288-6f31aa40ac44 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:22.323299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a1423-61ea-40a8-b42c-222804985566 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:22.355751 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:22.363383 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c67eb3-3a5e-40b0-9898-51acdbd7d0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:22.364689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c67eb3-3a5e-40b0-9898-51acdbd7d0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:22.365183 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c67eb3-3a5e-40b0-9898-51acdbd7d0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:22.365716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c67eb3-3a5e-40b0-9898-51acdbd7d0d9 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:22.367276 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c67eb3-3a5e-40b0-9898-51acdbd7d0d9 None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:22.369075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c67eb3-3a5e-40b0-9898-51acdbd7d0d9 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:22.371150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 80/166] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:22 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:22.373111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0cce3-9a33-46f3-ab28-508318eb9b8b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:22.373679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0cce3-9a33-46f3-ab28-508318eb9b8b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:22.374190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0cce3-9a33-46f3-ab28-508318eb9b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:22.393527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-95b0cce3-9a33-46f3-ab28-508318eb9b8b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:22.395043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-95b0cce3-9a33-46f3-ab28-508318eb9b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:22.400664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:22.419612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c03cea-ccd0-4afa-b288-6f31aa40ac44 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:22.456689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85714478-359c-438f-b57a-2ede03082c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:22.457495 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85714478-359c-438f-b57a-2ede03082c5c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:22.457779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85714478-359c-438f-b57a-2ede03082c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:22.458052 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85714478-359c-438f-b57a-2ede03082c5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:22.470793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85714478-359c-438f-b57a-2ede03082c5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:22.472968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 81/167] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:22.474196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0cce3-9a33-46f3-ab28-508318eb9b8b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-85714478-359c-438f-b57a-2ede03082c5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:22.499348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:22.500536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:22.501336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 87/168] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:06:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:22.504569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85714478-359c-438f-b57a-2ede03082c5c None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:22.589615 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0cce3-9a33-46f3-ab28-508318eb9b8b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:22.610972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85714478-359c-438f-b57a-2ede03082c5c None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:22.619202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:22.628469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea943d9-13b2-499f-ac4d-b4aa30211f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:22.629477 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea943d9-13b2-499f-ac4d-b4aa30211f1b None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:22.630019 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea943d9-13b2-499f-ac4d-b4aa30211f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:22.630496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea943d9-13b2-499f-ac4d-b4aa30211f1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:22.631538 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea943d9-13b2-499f-ac4d-b4aa30211f1b None admin] RBAC: Adding query filter params (role.id=22d7b181c7e041329c59c3eb6b492cd7, scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5, user.id=8fb531e37b334e7ea25ec28ef9d7d181) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:22.632049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea943d9-13b2-499f-ac4d-b4aa30211f1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:22.633828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea943d9-13b2-499f-ac4d-b4aa30211f1b None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:22.641773 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 82/169] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:22 2022] GET /identity/v3/role_assignments?user.id=8fb531e37b334e7ea25ec28ef9d7d181&scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5&role.id=22d7b181c7e041329c59c3eb6b492cd7 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:22.642921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea943d9-13b2-499f-ac4d-b4aa30211f1b None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:22.649568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb7e0a-477e-4379-b8ab-42a81601ac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:22.650664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb7e0a-477e-4379-b8ab-42a81601ac0e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:22.650947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb7e0a-477e-4379-b8ab-42a81601ac0e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:22.651222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb7e0a-477e-4379-b8ab-42a81601ac0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:22.664908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb7e0a-477e-4379-b8ab-42a81601ac0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-90fb7e0a-477e-4379-b8ab-42a81601ac0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:22.673455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:22.674429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:22.675198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 88/170] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:06:22.682911 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb7e0a-477e-4379-b8ab-42a81601ac0e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:22.708471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:22.759031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea943d9-13b2-499f-ac4d-b4aa30211f1b None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:22.796256 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8ebaf-4df2-4676-a4a3-772881e7be02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:22.797336 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8ebaf-4df2-4676-a4a3-772881e7be02 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:22.797827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8ebaf-4df2-4676-a4a3-772881e7be02 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:22.798305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8ebaf-4df2-4676-a4a3-772881e7be02 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:22.799261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8ebaf-4df2-4676-a4a3-772881e7be02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:22.799787 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8ebaf-4df2-4676-a4a3-772881e7be02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:22.801485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8ebaf-4df2-4676-a4a3-772881e7be02 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:22.810795 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 83/171] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:22.817265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8ebaf-4df2-4676-a4a3-772881e7be02 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:22.834959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb7e0a-477e-4379-b8ab-42a81601ac0e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:22.896542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16cb01-1182-4340-941c-c20f85cd58aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:22.897088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16cb01-1182-4340-941c-c20f85cd58aa None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:22.897373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16cb01-1182-4340-941c-c20f85cd58aa None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:22.897652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16cb01-1182-4340-941c-c20f85cd58aa None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:22.898406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16cb01-1182-4340-941c-c20f85cd58aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:22.898713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16cb01-1182-4340-941c-c20f85cd58aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:22.904252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16cb01-1182-4340-941c-c20f85cd58aa None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:22.918267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 89/172] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:22.922960 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:22.923803 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16cb01-1182-4340-941c-c20f85cd58aa None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:22.986567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8ebaf-4df2-4676-a4a3-772881e7be02 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:23.057702 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:23.074815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298918d-1943-47be-a203-471cf596456c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:23.076272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298918d-1943-47be-a203-471cf596456c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:23.076557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298918d-1943-47be-a203-471cf596456c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:23.076832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298918d-1943-47be-a203-471cf596456c None admin] PATH_INFO: `/v3/users/0bf6e418289c4b9c81b349d812dd7d25` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:23.081006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298918d-1943-47be-a203-471cf596456c None admin] RBAC: Authorizing `identity:get_user(user_id=0bf6e418289c4b9c81b349d812dd7d25)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:23.082567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298918d-1943-47be-a203-471cf596456c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:23.084720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 84/173] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:22 2022] GET /identity/v3/users/0bf6e418289c4b9c81b349d812dd7d25 => generated 247 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:23.090099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298918d-1943-47be-a203-471cf596456c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:23.096088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16cb01-1182-4340-941c-c20f85cd58aa None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:23.131298 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a0e0a-024b-405a-b353-1db19869f924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:23.132222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a0e0a-024b-405a-b353-1db19869f924 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:23.132506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a0e0a-024b-405a-b353-1db19869f924 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:23.132781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a0e0a-024b-405a-b353-1db19869f924 None admin] PATH_INFO: `/v3/users/8fb531e37b334e7ea25ec28ef9d7d181` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:23.133981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a0e0a-024b-405a-b353-1db19869f924 None admin] RBAC: Authorizing `identity:get_user(user_id=8fb531e37b334e7ea25ec28ef9d7d181)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:23.135444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a0e0a-024b-405a-b353-1db19869f924 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:23.138920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 90/174] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:22 2022] GET /identity/v3/users/8fb531e37b334e7ea25ec28ef9d7d181 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:23.143395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a0e0a-024b-405a-b353-1db19869f924 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:23.251772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a0e0a-024b-405a-b353-1db19869f924 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:23.289598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba7660-f572-4134-a52c-74afc14a452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:23.290261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba7660-f572-4134-a52c-74afc14a452b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:23.290547 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba7660-f572-4134-a52c-74afc14a452b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:23.290823 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba7660-f572-4134-a52c-74afc14a452b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:23.295626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298918d-1943-47be-a203-471cf596456c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:23.300130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:23.339056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:23.365826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af872f1c-bf7c-4c9c-b4c4-fbcb0d1e6b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:23.366499 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af872f1c-bf7c-4c9c-b4c4-fbcb0d1e6b49 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:23.366779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af872f1c-bf7c-4c9c-b4c4-fbcb0d1e6b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:23.367055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af872f1c-bf7c-4c9c-b4c4-fbcb0d1e6b49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:23.367761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba7660-f572-4134-a52c-74afc14a452b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:23.369254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba7660-f572-4134-a52c-74afc14a452b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:23.492179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 91/175] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 07:06:23.497701 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba7660-f572-4134-a52c-74afc14a452b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:23.521516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af872f1c-bf7c-4c9c-b4c4-fbcb0d1e6b49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:23.523324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af872f1c-bf7c-4c9c-b4c4-fbcb0d1e6b49 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:23.562394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:23.702444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba7660-f572-4134-a52c-74afc14a452b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:23.755610 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 85/176] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 07:06:23.761210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af872f1c-bf7c-4c9c-b4c4-fbcb0d1e6b49 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:23.791753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c5668-aee9-4421-ae74-b7b1497344c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:23.792584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c5668-aee9-4421-ae74-b7b1497344c0 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:23.792890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c5668-aee9-4421-ae74-b7b1497344c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:23.793408 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c5668-aee9-4421-ae74-b7b1497344c0 None admin] PATH_INFO: `/v3/projects/954126f281db499f89335d0beef28c3d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:23.795572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c5668-aee9-4421-ae74-b7b1497344c0 None admin] RBAC: Authorizing `identity:get_project(project_id=954126f281db499f89335d0beef28c3d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:23.797270 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c5668-aee9-4421-ae74-b7b1497344c0 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:23.799115 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 92/177] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:23 2022] GET /identity/v3/projects/954126f281db499f89335d0beef28c3d => generated 312 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:23.805065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d11a9b-b0d8-40d7-8ee2-050bde5a2b41 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:23.805338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d11a9b-b0d8-40d7-8ee2-050bde5a2b41 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:23.805615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d11a9b-b0d8-40d7-8ee2-050bde5a2b41 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:23.807349 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 93/178] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:23.809416 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d11a9b-b0d8-40d7-8ee2-050bde5a2b41 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:23.828451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:23.922884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d11a9b-b0d8-40d7-8ee2-050bde5a2b41 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:23.956458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:23.960239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34dd0c-2196-4f8d-a807-eedfb0ad3381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:23.961300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34dd0c-2196-4f8d-a807-eedfb0ad3381 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:23.961586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34dd0c-2196-4f8d-a807-eedfb0ad3381 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:23.961864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34dd0c-2196-4f8d-a807-eedfb0ad3381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:23.962705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34dd0c-2196-4f8d-a807-eedfb0ad3381 None admin] RBAC: Adding query filter params (role.id=ce36ba2b01744465803d0533b2a47592, scope.project.id=954126f281db499f89335d0beef28c3d, user.id=8fb531e37b334e7ea25ec28ef9d7d181) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:23.963017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34dd0c-2196-4f8d-a807-eedfb0ad3381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:23.964488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34dd0c-2196-4f8d-a807-eedfb0ad3381 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:23.971711 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 94/179] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:23 2022] GET /identity/v3/role_assignments?user.id=8fb531e37b334e7ea25ec28ef9d7d181&scope.project.id=954126f281db499f89335d0beef28c3d&role.id=ce36ba2b01744465803d0533b2a47592 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:23.977499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e1988-8dbb-484c-b840-08cc074dcfd9 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:23.977791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e1988-8dbb-484c-b840-08cc074dcfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:23.978065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e1988-8dbb-484c-b840-08cc074dcfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:24.011167 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-664e1988-8dbb-484c-b840-08cc074dcfd9 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:24.012531 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-664e1988-8dbb-484c-b840-08cc074dcfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:24.030375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af872f1c-bf7c-4c9c-b4c4-fbcb0d1e6b49 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:24.100678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cce63-7563-4c4a-9fce-89e4fd54a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:24.101531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cce63-7563-4c4a-9fce-89e4fd54a679 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:24.101825 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cce63-7563-4c4a-9fce-89e4fd54a679 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:24.102107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cce63-7563-4c4a-9fce-89e4fd54a679 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:24.105191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cce63-7563-4c4a-9fce-89e4fd54a679 None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:24.107342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cce63-7563-4c4a-9fce-89e4fd54a679 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:24.109343 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 86/180] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:23 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:24.115258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cce63-7563-4c4a-9fce-89e4fd54a679 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:24.134559 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 95/181] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:24.165668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e1988-8dbb-484c-b840-08cc074dcfd9 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:24.272487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:24.284626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cce63-7563-4c4a-9fce-89e4fd54a679 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:24.328999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdee5b1-594e-448c-ae39-0e22ed036ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:24.329784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdee5b1-594e-448c-ae39-0e22ed036ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:24.329982 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdee5b1-594e-448c-ae39-0e22ed036ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:24.330266 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdee5b1-594e-448c-ae39-0e22ed036ae1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:24.331274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdee5b1-594e-448c-ae39-0e22ed036ae1 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5, user.id=0bf6e418289c4b9c81b349d812dd7d25) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:24.331597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdee5b1-594e-448c-ae39-0e22ed036ae1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:24.333185 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdee5b1-594e-448c-ae39-0e22ed036ae1 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:24.347684 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 87/182] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:24 2022] GET /identity/v3/role_assignments?user.id=0bf6e418289c4b9c81b349d812dd7d25&scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:24.385116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e1988-8dbb-484c-b840-08cc074dcfd9 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:24.392595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:24.434125 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55714001-109a-4389-86c1-5964b26f16b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:24.435494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55714001-109a-4389-86c1-5964b26f16b4 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:24.435787 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55714001-109a-4389-86c1-5964b26f16b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:24.436087 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55714001-109a-4389-86c1-5964b26f16b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:24.448522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55714001-109a-4389-86c1-5964b26f16b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:24.456339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718daa3-6eee-4424-8d55-524cdf31335b None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:24.456644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718daa3-6eee-4424-8d55-524cdf31335b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:24.456931 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718daa3-6eee-4424-8d55-524cdf31335b None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-55714001-109a-4389-86c1-5964b26f16b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:24.457027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:24.457829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:24.458601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:24.458601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:24.458601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:24.458601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:24.458601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:24.458601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:24.458601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:24.458601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:24.458601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:24.458601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:24.458601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:24.458601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:24.459187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 88/184] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:06:24.459260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 96/184] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:06:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:24.465100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718daa3-6eee-4424-8d55-524cdf31335b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:24.465831 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0ccdc-ab67-41a4-a356-f5eb30238807 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:24.466119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0ccdc-ab67-41a4-a356-f5eb30238807 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:24.466398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0ccdc-ab67-41a4-a356-f5eb30238807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:24.486259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-07c0ccdc-ab67-41a4-a356-f5eb30238807 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:24.487483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-07c0ccdc-ab67-41a4-a356-f5eb30238807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:24.564161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 97/185] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:24.598493 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0ccdc-ab67-41a4-a356-f5eb30238807 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:24.664831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718daa3-6eee-4424-8d55-524cdf31335b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:24.709891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1a2f5-bb9f-45e7-9e02-61ce72fe4557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:24.710502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1a2f5-bb9f-45e7-9e02-61ce72fe4557 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:24.710786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1a2f5-bb9f-45e7-9e02-61ce72fe4557 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:24.711063 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1a2f5-bb9f-45e7-9e02-61ce72fe4557 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:24.711817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1a2f5-bb9f-45e7-9e02-61ce72fe4557 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:24.712198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1a2f5-bb9f-45e7-9e02-61ce72fe4557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:24.713693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1a2f5-bb9f-45e7-9e02-61ce72fe4557 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:24.730240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 89/186] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:24.731929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1a2f5-bb9f-45e7-9e02-61ce72fe4557 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:24.745047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:24.757467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:24.757809 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0ccdc-ab67-41a4-a356-f5eb30238807 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:24.806459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583f25a-01ea-4151-a88a-bbf63ba80752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:24.807625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583f25a-01ea-4151-a88a-bbf63ba80752 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:24.807913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583f25a-01ea-4151-a88a-bbf63ba80752 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:24.808230 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583f25a-01ea-4151-a88a-bbf63ba80752 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:24.810164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583f25a-01ea-4151-a88a-bbf63ba80752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:24.811620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583f25a-01ea-4151-a88a-bbf63ba80752 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:24.813730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 98/187] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:24.814915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71695e85-e1ae-4b39-ae38-3ea837b59474 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:24.815206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71695e85-e1ae-4b39-ae38-3ea837b59474 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:24.815483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71695e85-e1ae-4b39-ae38-3ea837b59474 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:24.817141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 99/188] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:24.822069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71695e85-e1ae-4b39-ae38-3ea837b59474 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:24.862412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1a2f5-bb9f-45e7-9e02-61ce72fe4557 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:24.900505 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad293fb6-e782-4d24-b6d4-5a5dd08c1ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:24.901394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad293fb6-e782-4d24-b6d4-5a5dd08c1ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:24.901675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad293fb6-e782-4d24-b6d4-5a5dd08c1ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:24.901945 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad293fb6-e782-4d24-b6d4-5a5dd08c1ab4 None admin] PATH_INFO: `/v3/users/ada5f7254b7147fc8c38de6c39727018` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:24.916523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad293fb6-e782-4d24-b6d4-5a5dd08c1ab4 None admin] RBAC: Authorizing `identity:get_user(user_id=ada5f7254b7147fc8c38de6c39727018)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:24.917955 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad293fb6-e782-4d24-b6d4-5a5dd08c1ab4 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:24.919853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 90/189] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:24 2022] GET /identity/v3/users/ada5f7254b7147fc8c38de6c39727018 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:24.925171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f517c-2ae3-4270-891c-290f98c735f2 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:24.925460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f517c-2ae3-4270-891c-290f98c735f2 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:24.925736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f517c-2ae3-4270-891c-290f98c735f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:24.937616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71695e85-e1ae-4b39-ae38-3ea837b59474 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:24.943122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-4f1f517c-2ae3-4270-891c-290f98c735f2 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:24.944328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-4f1f517c-2ae3-4270-891c-290f98c735f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:24.974448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6c150-47fe-4b58-b2eb-d52c2391b127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:24.975126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6c150-47fe-4b58-b2eb-d52c2391b127 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:24.975415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6c150-47fe-4b58-b2eb-d52c2391b127 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:24.975698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6c150-47fe-4b58-b2eb-d52c2391b127 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:24.976436 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6c150-47fe-4b58-b2eb-d52c2391b127 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:24.977883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6c150-47fe-4b58-b2eb-d52c2391b127 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:24.999114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 100/190] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:06:24 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:06:24.999989 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6c150-47fe-4b58-b2eb-d52c2391b127 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:25.019698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:25.026016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 91/191] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:25.054506 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f517c-2ae3-4270-891c-290f98c735f2 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:25.057913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:25.129413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6c150-47fe-4b58-b2eb-d52c2391b127 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:25.169368 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5a20b-600a-4708-94d0-7042dddcf8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:25.169955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5a20b-600a-4708-94d0-7042dddcf8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:25.170239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5a20b-600a-4708-94d0-7042dddcf8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:25.170517 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5a20b-600a-4708-94d0-7042dddcf8f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:25.170697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f517c-2ae3-4270-891c-290f98c735f2 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:25.214880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900212e-f649-45e5-b0fd-e3b598880017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:25.216953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900212e-f649-45e5-b0fd-e3b598880017 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:25.216953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900212e-f649-45e5-b0fd-e3b598880017 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:25.216953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900212e-f649-45e5-b0fd-e3b598880017 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:25.230165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900212e-f649-45e5-b0fd-e3b598880017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-4900212e-f649-45e5-b0fd-e3b598880017 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:25.238488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:25.239407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:25.240289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 92/192] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:06:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:06:25.245654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900212e-f649-45e5-b0fd-e3b598880017 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:25.257307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5a20b-600a-4708-94d0-7042dddcf8f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:25.259520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5a20b-600a-4708-94d0-7042dddcf8f6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:25.280124 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:25.337843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:25.384732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900212e-f649-45e5-b0fd-e3b598880017 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:25.390542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 101/193] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:25.395504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed4046-6136-4ebd-8f1e-89e481db4488 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:25.395645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed4046-6136-4ebd-8f1e-89e481db4488 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:25.395872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed4046-6136-4ebd-8f1e-89e481db4488 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:25.397266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 102/194] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:25.398069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed4046-6136-4ebd-8f1e-89e481db4488 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:25.440711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d2d4d-d5c7-4960-9501-fc7cbd75cb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:25.441399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d2d4d-d5c7-4960-9501-fc7cbd75cb62 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:25.441774 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d2d4d-d5c7-4960-9501-fc7cbd75cb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:25.442200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d2d4d-d5c7-4960-9501-fc7cbd75cb62 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:25.442716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d2d4d-d5c7-4960-9501-fc7cbd75cb62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:25.443035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d2d4d-d5c7-4960-9501-fc7cbd75cb62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:25.444560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d2d4d-d5c7-4960-9501-fc7cbd75cb62 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:25.454627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 93/195] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:06:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:25.455890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3772ade-9112-436d-a3ab-5f07805831b7 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:25.456250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3772ade-9112-436d-a3ab-5f07805831b7 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:25.456535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3772ade-9112-436d-a3ab-5f07805831b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:25.476855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-e3772ade-9112-436d-a3ab-5f07805831b7 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:25.478201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e3772ade-9112-436d-a3ab-5f07805831b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:25.535738 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed4046-6136-4ebd-8f1e-89e481db4488 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:25.551850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 94/196] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:25.552876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3772ade-9112-436d-a3ab-5f07805831b7 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:25.554797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:25.586083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65e680-7654-4960-9821-1f67387b8046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:25.586957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65e680-7654-4960-9821-1f67387b8046 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:25.587244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65e680-7654-4960-9821-1f67387b8046 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:25.587524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65e680-7654-4960-9821-1f67387b8046 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:25.588737 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65e680-7654-4960-9821-1f67387b8046 None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:25.590219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65e680-7654-4960-9821-1f67387b8046 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:25.592728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 103/197] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:25 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:25.593514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65e680-7654-4960-9821-1f67387b8046 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:25.628330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:25.677024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3772ade-9112-436d-a3ab-5f07805831b7 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:25.718234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe091f-3b98-4969-bda8-74e26c326457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:25.719040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe091f-3b98-4969-bda8-74e26c326457 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:25.719326 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe091f-3b98-4969-bda8-74e26c326457 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:25.719607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe091f-3b98-4969-bda8-74e26c326457 None admin] PATH_INFO: `/v3/users/ada5f7254b7147fc8c38de6c39727018` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:25.721053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe091f-3b98-4969-bda8-74e26c326457 None admin] RBAC: Authorizing `identity:get_user(user_id=ada5f7254b7147fc8c38de6c39727018)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:25.722521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe091f-3b98-4969-bda8-74e26c326457 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:25.723544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65e680-7654-4960-9821-1f67387b8046 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:25.724505 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 95/198] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:25 2022] GET /identity/v3/users/ada5f7254b7147fc8c38de6c39727018 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:25.725448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe091f-3b98-4969-bda8-74e26c326457 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:25.768120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4792c6-cc74-4dea-93c5-0df5a87d4fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:25.768931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4792c6-cc74-4dea-93c5-0df5a87d4fff None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:25.769218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4792c6-cc74-4dea-93c5-0df5a87d4fff None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:25.769499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4792c6-cc74-4dea-93c5-0df5a87d4fff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:25.784157 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4792c6-cc74-4dea-93c5-0df5a87d4fff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-db4792c6-cc74-4dea-93c5-0df5a87d4fff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:25.795682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:25.796583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:25.797379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:25.797379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:25.797379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:25.797379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:25.797379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:25.797379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:25.797379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:25.797379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:25.797379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:25.797379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:25.797379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:25.797379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:25.800153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 104/199] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:25.800993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4792c6-cc74-4dea-93c5-0df5a87d4fff None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:25.839636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe091f-3b98-4969-bda8-74e26c326457 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:25.857799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:25.870429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:25.888940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31768471-ea58-4fe1-8803-7df7600edc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:25.892005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31768471-ea58-4fe1-8803-7df7600edc61 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:25.892564 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31768471-ea58-4fe1-8803-7df7600edc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:25.893026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31768471-ea58-4fe1-8803-7df7600edc61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:25.894168 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31768471-ea58-4fe1-8803-7df7600edc61 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d, user.id=ada5f7254b7147fc8c38de6c39727018) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:25.894666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31768471-ea58-4fe1-8803-7df7600edc61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:25.896466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31768471-ea58-4fe1-8803-7df7600edc61 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:25.902853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 96/200] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:25 2022] GET /identity/v3/role_assignments?user.id=ada5f7254b7147fc8c38de6c39727018&scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:25.909418 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31768471-ea58-4fe1-8803-7df7600edc61 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:25.940852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4792c6-cc74-4dea-93c5-0df5a87d4fff None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:25.984788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe25acb-6001-4b28-8793-88b0b81aea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:25.986413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe25acb-6001-4b28-8793-88b0b81aea03 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:25.986717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe25acb-6001-4b28-8793-88b0b81aea03 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:25.987003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe25acb-6001-4b28-8793-88b0b81aea03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:26.108611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31768471-ea58-4fe1-8803-7df7600edc61 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:26.144885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe25acb-6001-4b28-8793-88b0b81aea03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:26.146655 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe25acb-6001-4b28-8793-88b0b81aea03 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:26.146761 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11deb6-158e-4be2-9415-3b33098bd121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:26.147344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11deb6-158e-4be2-9415-3b33098bd121 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:26.147629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11deb6-158e-4be2-9415-3b33098bd121 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:26.147907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11deb6-158e-4be2-9415-3b33098bd121 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:26.148689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11deb6-158e-4be2-9415-3b33098bd121 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:26.149001 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11deb6-158e-4be2-9415-3b33098bd121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:26.150990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11deb6-158e-4be2-9415-3b33098bd121 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:26.161613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 97/201] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:26.168367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11deb6-158e-4be2-9415-3b33098bd121 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:26.232225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:26.247859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:26.308384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11deb6-158e-4be2-9415-3b33098bd121 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:26.345713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8f79d-7c65-45c5-89e4-6e51fcb91b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:26.347495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8f79d-7c65-45c5-89e4-6e51fcb91b2d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:26.348103 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8f79d-7c65-45c5-89e4-6e51fcb91b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:26.348718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8f79d-7c65-45c5-89e4-6e51fcb91b2d None admin] PATH_INFO: `/v3/users/7b5dec3a875348d9839b5ccf4cc3b9c5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:26.350327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8f79d-7c65-45c5-89e4-6e51fcb91b2d None admin] RBAC: Authorizing `identity:get_user(user_id=7b5dec3a875348d9839b5ccf4cc3b9c5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:26.352299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8f79d-7c65-45c5-89e4-6e51fcb91b2d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:26.354192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 98/202] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:26 2022] GET /identity/v3/users/7b5dec3a875348d9839b5ccf4cc3b9c5 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:26.360115 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 105/203] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:26.361339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8f79d-7c65-45c5-89e4-6e51fcb91b2d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:26.367292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe25acb-6001-4b28-8793-88b0b81aea03 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:26.515495 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe25acb-6001-4b28-8793-88b0b81aea03 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:26.550932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8f79d-7c65-45c5-89e4-6e51fcb91b2d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:26.560856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281db86d-56ce-4f48-830f-5ea98d041ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:26.561732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281db86d-56ce-4f48-830f-5ea98d041ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:26.562017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281db86d-56ce-4f48-830f-5ea98d041ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:26.562289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281db86d-56ce-4f48-830f-5ea98d041ff2 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:26.565904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281db86d-56ce-4f48-830f-5ea98d041ff2 None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:26.567606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281db86d-56ce-4f48-830f-5ea98d041ff2 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:26.571431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 106/204] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:26 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:26.581892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281db86d-56ce-4f48-830f-5ea98d041ff2 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:26.593938 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:26.605460 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:26.627362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2a4e2-466b-49ec-a2b4-87fbdc603208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:26.628030 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2a4e2-466b-49ec-a2b4-87fbdc603208 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:26.628339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2a4e2-466b-49ec-a2b4-87fbdc603208 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:26.628616 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2a4e2-466b-49ec-a2b4-87fbdc603208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:26.757887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2a4e2-466b-49ec-a2b4-87fbdc603208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:26.760392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2a4e2-466b-49ec-a2b4-87fbdc603208 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:26.772254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281db86d-56ce-4f48-830f-5ea98d041ff2 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:26.837511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ee661-d827-401a-a541-a4afba5e9113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:26.838602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ee661-d827-401a-a541-a4afba5e9113 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:26.839090 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ee661-d827-401a-a541-a4afba5e9113 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:26.839568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ee661-d827-401a-a541-a4afba5e9113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:26.842489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ee661-d827-401a-a541-a4afba5e9113 None admin] RBAC: Adding query filter params (role.id=22d7b181c7e041329c59c3eb6b492cd7, scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d, user.id=ada5f7254b7147fc8c38de6c39727018) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:26.843026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ee661-d827-401a-a541-a4afba5e9113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:26.844737 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ee661-d827-401a-a541-a4afba5e9113 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:26.859045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 107/205] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:26 2022] GET /identity/v3/role_assignments?user.id=ada5f7254b7147fc8c38de6c39727018&scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d&role.id=22d7b181c7e041329c59c3eb6b492cd7 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:26.915616 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 99/206] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:26.925271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ee661-d827-401a-a541-a4afba5e9113 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:27.038495 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:27.072703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ee661-d827-401a-a541-a4afba5e9113 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:27.120873 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e3c34-80c8-4128-aa7d-f75f020a5f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:27.121765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e3c34-80c8-4128-aa7d-f75f020a5f9d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:27.122587 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e3c34-80c8-4128-aa7d-f75f020a5f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:27.122867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e3c34-80c8-4128-aa7d-f75f020a5f9d None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:27.124136 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e3c34-80c8-4128-aa7d-f75f020a5f9d None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:27.125594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e3c34-80c8-4128-aa7d-f75f020a5f9d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:27.127606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 108/207] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:26 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:27.133543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2a4e2-466b-49ec-a2b4-87fbdc603208 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:27.164307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:27.253150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2a4e2-466b-49ec-a2b4-87fbdc603208 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:27.289205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be6716-f835-43ae-9bee-a8a4b626fe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:27.289796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be6716-f835-43ae-9bee-a8a4b626fe36 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:27.290088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be6716-f835-43ae-9bee-a8a4b626fe36 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:27.290373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be6716-f835-43ae-9bee-a8a4b626fe36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:27.291216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be6716-f835-43ae-9bee-a8a4b626fe36 None admin] RBAC: Adding query filter params (role.id=b3b7dfbee8c34760b5acbb12576c8fdd, scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5, user.id=7b5dec3a875348d9839b5ccf4cc3b9c5) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:27.291537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be6716-f835-43ae-9bee-a8a4b626fe36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:27.293023 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be6716-f835-43ae-9bee-a8a4b626fe36 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:27.305248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 100/208] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:27 2022] GET /identity/v3/role_assignments?user.id=7b5dec3a875348d9839b5ccf4cc3b9c5&scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5&role.id=b3b7dfbee8c34760b5acbb12576c8fdd => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:27.632139 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a9140-fc2f-4219-aa5c-bde8316dec1d None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:27.632639 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a9140-fc2f-4219-aa5c-bde8316dec1d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:27.632639 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a9140-fc2f-4219-aa5c-bde8316dec1d None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:27.633998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 109/209] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:27.639454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bede29-d7e5-4a91-bacb-7a2c4ac23d7d None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:27.639859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bede29-d7e5-4a91-bacb-7a2c4ac23d7d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:27.640057 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bede29-d7e5-4a91-bacb-7a2c4ac23d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:27.659655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-e1bede29-d7e5-4a91-bacb-7a2c4ac23d7d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:27.660949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e1bede29-d7e5-4a91-bacb-7a2c4ac23d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:27.662245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e920964-d587-49a2-be49-8cf6830216d8 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:27.662532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e920964-d587-49a2-be49-8cf6830216d8 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:27.662816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e920964-d587-49a2-be49-8cf6830216d8 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:27.665543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 110/210] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:27.672674 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3946a-75d6-4976-a85e-59de7f5f9baa None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:27.672964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3946a-75d6-4976-a85e-59de7f5f9baa None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:27.673242 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3946a-75d6-4976-a85e-59de7f5f9baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:27.695708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-8ba3946a-75d6-4976-a85e-59de7f5f9baa None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:27.697339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-8ba3946a-75d6-4976-a85e-59de7f5f9baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:27.757795 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 101/211] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:27.782443 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bede29-d7e5-4a91-bacb-7a2c4ac23d7d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:27.787393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 111/212] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:27.796705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:27.818941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3946a-75d6-4976-a85e-59de7f5f9baa None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:27.901288 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bede29-d7e5-4a91-bacb-7a2c4ac23d7d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:27.910895 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:27.941203 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fe11e-21ab-4c28-af10-9ef2557d5592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:27.942467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fe11e-21ab-4c28-af10-9ef2557d5592 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:27.942977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fe11e-21ab-4c28-af10-9ef2557d5592 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:27.943449 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fe11e-21ab-4c28-af10-9ef2557d5592 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:27.947947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3946a-75d6-4976-a85e-59de7f5f9baa None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:27.955639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fe11e-21ab-4c28-af10-9ef2557d5592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-239fe11e-21ab-4c28-af10-9ef2557d5592 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:27.964521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:27.965403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:27.966230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:27.966230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:27.966230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:27.966230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:27.966230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:27.966230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:27.966230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:27.966230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:27.966230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:27.966230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:27.966230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:27.966230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:27.968996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 102/213] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:06:27.975365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fe11e-21ab-4c28-af10-9ef2557d5592 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:27.991321 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c41914-8011-4ed8-9489-f4ce430bbded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:27.992159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c41914-8011-4ed8-9489-f4ce430bbded None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:27.992449 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c41914-8011-4ed8-9489-f4ce430bbded None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:27.992732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c41914-8011-4ed8-9489-f4ce430bbded None admin] PATH_INFO: `/v3/domains/default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:27.994040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c41914-8011-4ed8-9489-f4ce430bbded None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:27.996197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c41914-8011-4ed8-9489-f4ce430bbded None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:28.002819 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 112/214] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:28.003965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c41914-8011-4ed8-9489-f4ce430bbded None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:28.061134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:28.096595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fe11e-21ab-4c28-af10-9ef2557d5592 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:28.126094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c41914-8011-4ed8-9489-f4ce430bbded None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:28.134669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60c0e5-350e-461a-86af-e38d1a7d3b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:28.135914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c0e5-350e-461a-86af-e38d1a7d3b31 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:28.136480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c0e5-350e-461a-86af-e38d1a7d3b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:28.136971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c0e5-350e-461a-86af-e38d1a7d3b31 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:28.138505 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60c0e5-350e-461a-86af-e38d1a7d3b31 None admin] RBAC: Adding query filter params (name=member) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:28.139111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60c0e5-350e-461a-86af-e38d1a7d3b31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:28.140925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60c0e5-350e-461a-86af-e38d1a7d3b31 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:28.151854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 103/215] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:28.157393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60c0e5-350e-461a-86af-e38d1a7d3b31 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:28.173606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9ddf3-5e03-44f0-846d-32032fb54248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:28.174285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9ddf3-5e03-44f0-846d-32032fb54248 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:28.174573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9ddf3-5e03-44f0-846d-32032fb54248 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:28.174852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9ddf3-5e03-44f0-846d-32032fb54248 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:28.175572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9ddf3-5e03-44f0-846d-32032fb54248 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:28.177593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9ddf3-5e03-44f0-846d-32032fb54248 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:28.189378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:28.208411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 113/216] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:06:28 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:28.284872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60c0e5-350e-461a-86af-e38d1a7d3b31 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:28.326107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d0442-da8f-4854-a9da-223152d126cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:28.327030 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d0442-da8f-4854-a9da-223152d126cb None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:28.327310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d0442-da8f-4854-a9da-223152d126cb None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:28.327586 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d0442-da8f-4854-a9da-223152d126cb None admin] PATH_INFO: `/v3/users/dddd8787eb3643e2a6573146d6d8846f` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:28.330084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:28.345497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d0442-da8f-4854-a9da-223152d126cb None admin] RBAC: Authorizing `identity:get_user(user_id=dddd8787eb3643e2a6573146d6d8846f)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:28.347641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d0442-da8f-4854-a9da-223152d126cb None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:28.350354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 104/217] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:28 2022] GET /identity/v3/users/dddd8787eb3643e2a6573146d6d8846f => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:28.356369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9ddf3-5e03-44f0-846d-32032fb54248 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:28.400956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259e77c-b80c-403d-9d35-b9536b3ac6b0 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:28.401553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259e77c-b80c-403d-9d35-b9536b3ac6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:28.402114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259e77c-b80c-403d-9d35-b9536b3ac6b0 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:28.403605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 105/218] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:06:28.409344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71dfee-12c9-460a-a937-d70555ab5717 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:28.409711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71dfee-12c9-460a-a937-d70555ab5717 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:28.409924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71dfee-12c9-460a-a937-d70555ab5717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:28.429214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-ec71dfee-12c9-460a-a937-d70555ab5717 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:28.430519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ec71dfee-12c9-460a-a937-d70555ab5717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:28.476626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9ddf3-5e03-44f0-846d-32032fb54248 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:28.513300 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 106/219] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:28.516506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a10b3-0a1e-41cd-baf8-d840f2988d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:28.517167 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a10b3-0a1e-41cd-baf8-d840f2988d8f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:28.517469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a10b3-0a1e-41cd-baf8-d840f2988d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:28.517748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a10b3-0a1e-41cd-baf8-d840f2988d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:28.540973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71dfee-12c9-460a-a937-d70555ab5717 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:28.602806 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:28.615773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a10b3-0a1e-41cd-baf8-d840f2988d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:28.616024 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a10b3-0a1e-41cd-baf8-d840f2988d8f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:28.660455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71dfee-12c9-460a-a937-d70555ab5717 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:28.686597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:28.701954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4f71b-8ac8-4608-b6e1-9023dfeece92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:28.703031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4f71b-8ac8-4608-b6e1-9023dfeece92 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:28.703320 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4f71b-8ac8-4608-b6e1-9023dfeece92 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:28.703584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4f71b-8ac8-4608-b6e1-9023dfeece92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:28.717234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4f71b-8ac8-4608-b6e1-9023dfeece92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-b9a4f71b-8ac8-4608-b6e1-9023dfeece92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:28.727464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:28.728381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:28.729178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 107/220] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:06:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:06:28.737095 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4f71b-8ac8-4608-b6e1-9023dfeece92 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:28.758048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 114/221] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:28.764182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a10b3-0a1e-41cd-baf8-d840f2988d8f None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:28.879783 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4f71b-8ac8-4608-b6e1-9023dfeece92 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:28.916845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaebfa1-fc18-42ae-b727-2471c1b1061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:28.917815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaebfa1-fc18-42ae-b727-2471c1b1061e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:28.918279 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaebfa1-fc18-42ae-b727-2471c1b1061e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:28.918751 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaebfa1-fc18-42ae-b727-2471c1b1061e None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:28.920243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaebfa1-fc18-42ae-b727-2471c1b1061e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:28.920730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaebfa1-fc18-42ae-b727-2471c1b1061e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:28.922410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaebfa1-fc18-42ae-b727-2471c1b1061e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:28.932261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 108/222] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:28.941164 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaebfa1-fc18-42ae-b727-2471c1b1061e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:28.970085 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:28.985142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a10b3-0a1e-41cd-baf8-d840f2988d8f None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:29.008273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:29.067657 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf1f84-2d35-4925-92f2-9c8ddb5b0b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:29.068570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf1f84-2d35-4925-92f2-9c8ddb5b0b9e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:29.068853 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf1f84-2d35-4925-92f2-9c8ddb5b0b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:29.069176 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf1f84-2d35-4925-92f2-9c8ddb5b0b9e None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:29.070315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf1f84-2d35-4925-92f2-9c8ddb5b0b9e None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:29.071783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf1f84-2d35-4925-92f2-9c8ddb5b0b9e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:29.073466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 115/223] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:28 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:29.080126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf1f84-2d35-4925-92f2-9c8ddb5b0b9e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:29.082979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaebfa1-fc18-42ae-b727-2471c1b1061e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:29.138445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5dec32-a5be-4604-b34b-2516e59466a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:29.139329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5dec32-a5be-4604-b34b-2516e59466a9 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:29.139612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5dec32-a5be-4604-b34b-2516e59466a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:29.139888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5dec32-a5be-4604-b34b-2516e59466a9 None admin] PATH_INFO: `/v3/users/ada5f7254b7147fc8c38de6c39727018` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:29.141122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5dec32-a5be-4604-b34b-2516e59466a9 None admin] RBAC: Authorizing `identity:get_user(user_id=ada5f7254b7147fc8c38de6c39727018)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:29.142569 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5dec32-a5be-4604-b34b-2516e59466a9 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:29.144522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 109/224] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:28 2022] GET /identity/v3/users/ada5f7254b7147fc8c38de6c39727018 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:29.149794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5dec32-a5be-4604-b34b-2516e59466a9 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:29.212950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf1f84-2d35-4925-92f2-9c8ddb5b0b9e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:29.261071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5dec32-a5be-4604-b34b-2516e59466a9 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:29.276755 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:29.283176 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf5523-6568-4000-9c52-12352fc2288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:29.284088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf5523-6568-4000-9c52-12352fc2288a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:29.284571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf5523-6568-4000-9c52-12352fc2288a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:29.285025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf5523-6568-4000-9c52-12352fc2288a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:29.292088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf5523-6568-4000-9c52-12352fc2288a None admin] RBAC: Adding query filter params (role.id=ce36ba2b01744465803d0533b2a47592, scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d, user.id=dddd8787eb3643e2a6573146d6d8846f) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:29.292580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf5523-6568-4000-9c52-12352fc2288a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:29.295143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf5523-6568-4000-9c52-12352fc2288a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:29.300096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ec8e9-ffa2-4ae0-9395-e99864a1a9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:29.300983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ec8e9-ffa2-4ae0-9395-e99864a1a9ac None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:29.301274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ec8e9-ffa2-4ae0-9395-e99864a1a9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:29.301557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ec8e9-ffa2-4ae0-9395-e99864a1a9ac None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:29.302775 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ec8e9-ffa2-4ae0-9395-e99864a1a9ac None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:29.304281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ec8e9-ffa2-4ae0-9395-e99864a1a9ac None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:29.306643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 110/225] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:29 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:29.307260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 116/226] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:29 2022] GET /identity/v3/role_assignments?user.id=dddd8787eb3643e2a6573146d6d8846f&scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d&role.id=ce36ba2b01744465803d0533b2a47592 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:29.312522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ec8e9-ffa2-4ae0-9395-e99864a1a9ac None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:29.397634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6309275-ad23-41e2-92ec-5b50181f58bf None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:29.397925 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6309275-ad23-41e2-92ec-5b50181f58bf None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:29.398213 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6309275-ad23-41e2-92ec-5b50181f58bf None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:29.400102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 117/227] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:29.405636 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965df41-3deb-4110-b413-8d97f6590539 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:29.405929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965df41-3deb-4110-b413-8d97f6590539 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:29.406210 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965df41-3deb-4110-b413-8d97f6590539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:29.426648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c965df41-3deb-4110-b413-8d97f6590539 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:29.427868 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c965df41-3deb-4110-b413-8d97f6590539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:29.458757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:29.482821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ec8e9-ffa2-4ae0-9395-e99864a1a9ac None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:29.510343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 118/228] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:29.536375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ff59f-e5fa-4358-8532-a3085942ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:29.537433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ff59f-e5fa-4358-8532-a3085942ad88 None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:29.537871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ff59f-e5fa-4358-8532-a3085942ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:29.538243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ff59f-e5fa-4358-8532-a3085942ad88 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d/users/ada5f7254b7147fc8c38de6c39727018/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:29.540437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ff59f-e5fa-4358-8532-a3085942ad88 None admin] RBAC: Authorizing `identity:create_grant(project_id=36fac836e8de4d52aa468bc5deaf2a2d, user_id=ada5f7254b7147fc8c38de6c39727018, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:29.542057 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ff59f-e5fa-4358-8532-a3085942ad88 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:29.547626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965df41-3deb-4110-b413-8d97f6590539 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:29.564565 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 111/229] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:06:29 2022] PUT /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d/users/ada5f7254b7147fc8c38de6c39727018/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:29.665913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965df41-3deb-4110-b413-8d97f6590539 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:29.703165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f7d5d-4059-47e5-9659-7c0eadced634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:29.703985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f7d5d-4059-47e5-9659-7c0eadced634 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:29.704302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f7d5d-4059-47e5-9659-7c0eadced634 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:29.704584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f7d5d-4059-47e5-9659-7c0eadced634 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:29.716990 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f7d5d-4059-47e5-9659-7c0eadced634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-8c0f7d5d-4059-47e5-9659-7c0eadced634 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:29.725568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:29.726523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:29.727316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 119/230] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:06:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:29.734858 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ff59f-e5fa-4358-8532-a3085942ad88 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:29.838504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:29.916753 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ff59f-e5fa-4358-8532-a3085942ad88 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:29.958652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84601cda-ed50-4046-ad49-d3fe209a3fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:29.959675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84601cda-ed50-4046-ad49-d3fe209a3fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:29.960178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84601cda-ed50-4046-ad49-d3fe209a3fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:29.960676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84601cda-ed50-4046-ad49-d3fe209a3fe3 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:29.961656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84601cda-ed50-4046-ad49-d3fe209a3fe3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:29.962174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84601cda-ed50-4046-ad49-d3fe209a3fe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:29.963758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84601cda-ed50-4046-ad49-d3fe209a3fe3 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:29.975997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 112/231] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:06:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:29.982325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f7d5d-4059-47e5-9659-7c0eadced634 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:30.010267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:30.107884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f7d5d-4059-47e5-9659-7c0eadced634 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:30.143502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f12059-f046-4060-beaa-54458bca4a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:30.144412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f12059-f046-4060-beaa-54458bca4a81 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:30.144745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f12059-f046-4060-beaa-54458bca4a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:30.145025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f12059-f046-4060-beaa-54458bca4a81 None admin] PATH_INFO: `/v3/users/ada5f7254b7147fc8c38de6c39727018` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:30.146210 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f12059-f046-4060-beaa-54458bca4a81 None admin] RBAC: Authorizing `identity:get_user(user_id=ada5f7254b7147fc8c38de6c39727018)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:30.147653 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f12059-f046-4060-beaa-54458bca4a81 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:30.149556 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 120/232] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:29 2022] GET /identity/v3/users/ada5f7254b7147fc8c38de6c39727018 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:30.155960 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84601cda-ed50-4046-ad49-d3fe209a3fe3 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:30.308443 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84601cda-ed50-4046-ad49-d3fe209a3fe3 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:30.310339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:30.360096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34aa93-15aa-45f8-9c2f-94afd068df66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:30.360980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34aa93-15aa-45f8-9c2f-94afd068df66 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:30.361264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34aa93-15aa-45f8-9c2f-94afd068df66 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:30.361542 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34aa93-15aa-45f8-9c2f-94afd068df66 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:30.362756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34aa93-15aa-45f8-9c2f-94afd068df66 None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:30.364241 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34aa93-15aa-45f8-9c2f-94afd068df66 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:30.366270 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 113/233] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:30 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:30.371786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f12059-f046-4060-beaa-54458bca4a81 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:30.515665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:30.534089 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f12059-f046-4060-beaa-54458bca4a81 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:30.572944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d597e-441a-42bf-9830-370435e9cfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:30.573860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d597e-441a-42bf-9830-370435e9cfb8 None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:30.574144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d597e-441a-42bf-9830-370435e9cfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:30.574420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d597e-441a-42bf-9830-370435e9cfb8 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d/users/ada5f7254b7147fc8c38de6c39727018/roles/22d7b181c7e041329c59c3eb6b492cd7` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:30.577217 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d597e-441a-42bf-9830-370435e9cfb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=36fac836e8de4d52aa468bc5deaf2a2d, user_id=ada5f7254b7147fc8c38de6c39727018, role_id=22d7b181c7e041329c59c3eb6b492cd7)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:30.578660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d597e-441a-42bf-9830-370435e9cfb8 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:30.593787 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 121/234] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:06:30 2022] PUT /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d/users/ada5f7254b7147fc8c38de6c39727018/roles/22d7b181c7e041329c59c3eb6b492cd7 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:31.088542 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a69349-7623-4b55-b383-241b3f03c16d None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:31.089416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a69349-7623-4b55-b383-241b3f03c16d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:31.089840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a69349-7623-4b55-b383-241b3f03c16d None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:31.092002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 114/235] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:31.098012 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d350ce-2a66-4e57-9b83-a574e20c8f73 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:31.098976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d350ce-2a66-4e57-9b83-a574e20c8f73 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:31.099463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d350ce-2a66-4e57-9b83-a574e20c8f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:31.120321 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-73d350ce-2a66-4e57-9b83-a574e20c8f73 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:31.121823 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-73d350ce-2a66-4e57-9b83-a574e20c8f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:31.213429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 122/236] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:31.239837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0e83b-294d-4ee7-97f1-fdef9e01ebca None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:31.240483 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0e83b-294d-4ee7-97f1-fdef9e01ebca None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:31.240978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0e83b-294d-4ee7-97f1-fdef9e01ebca None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:31.242784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 115/237] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:31.248619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6e17-723a-4bfe-82ff-ec762c10236c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:31.249122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6e17-723a-4bfe-82ff-ec762c10236c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:31.249610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6e17-723a-4bfe-82ff-ec762c10236c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:31.260048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0e83b-294d-4ee7-97f1-fdef9e01ebca None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:31.271433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ac5b6e17-723a-4bfe-82ff-ec762c10236c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:31.272889 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ac5b6e17-723a-4bfe-82ff-ec762c10236c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:31.397799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 123/238] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:31.443641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0e83b-294d-4ee7-97f1-fdef9e01ebca None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:31.451530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b6e17-723a-4bfe-82ff-ec762c10236c None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:31.463216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:31.487370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9a039-52b9-4b68-a84c-4899003dcdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:31.488629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9a039-52b9-4b68-a84c-4899003dcdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:31.489110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9a039-52b9-4b68-a84c-4899003dcdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:31.489606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9a039-52b9-4b68-a84c-4899003dcdc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:31.490970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9a039-52b9-4b68-a84c-4899003dcdc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:31.492889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9a039-52b9-4b68-a84c-4899003dcdc8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:31.494877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 116/239] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:31.500940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9a039-52b9-4b68-a84c-4899003dcdc8 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:31.546830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:31.658186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b6e17-723a-4bfe-82ff-ec762c10236c None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:31.676718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9a039-52b9-4b68-a84c-4899003dcdc8 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:31.698146 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aa691-06a1-4092-b4c8-817ae1508109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:31.698974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aa691-06a1-4092-b4c8-817ae1508109 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:31.699253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aa691-06a1-4092-b4c8-817ae1508109 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:31.699522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aa691-06a1-4092-b4c8-817ae1508109 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:31.711306 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47aa691-06a1-4092-b4c8-817ae1508109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-d47aa691-06a1-4092-b4c8-817ae1508109 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:31.721109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:31.722013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:31.722802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 124/240] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:06:31.730322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aa691-06a1-4092-b4c8-817ae1508109 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:31.753715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3993800-f433-4f06-957c-47b361ecddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:31.754708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3993800-f433-4f06-957c-47b361ecddb9 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:31.755219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3993800-f433-4f06-957c-47b361ecddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:31.755687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3993800-f433-4f06-957c-47b361ecddb9 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:31.756625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3993800-f433-4f06-957c-47b361ecddb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:31.764086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3993800-f433-4f06-957c-47b361ecddb9 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:31.801076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 117/241] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:06:31 2022] POST /identity/v3/users => generated 293 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:31.806590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:31.847282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aa691-06a1-4092-b4c8-817ae1508109 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:31.885068 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642d29c-06f0-4399-a004-1d736ee14ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:31.886022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642d29c-06f0-4399-a004-1d736ee14ffa None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:31.886494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642d29c-06f0-4399-a004-1d736ee14ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:31.886939 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642d29c-06f0-4399-a004-1d736ee14ffa None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:31.887883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642d29c-06f0-4399-a004-1d736ee14ffa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:31.888392 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642d29c-06f0-4399-a004-1d736ee14ffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:31.889929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642d29c-06f0-4399-a004-1d736ee14ffa None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:31.900168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 125/242] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:31.908111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3993800-f433-4f06-957c-47b361ecddb9 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:32.038191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a31e3e-b182-4776-a0a5-24736ceef34a None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:32.038482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a31e3e-b182-4776-a0a5-24736ceef34a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:32.038767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a31e3e-b182-4776-a0a5-24736ceef34a None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:32.044863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 126/243] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:32.050460 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff086331-a172-4f76-be5c-7c63d7ef8ef6 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:32.050795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff086331-a172-4f76-be5c-7c63d7ef8ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:32.051025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff086331-a172-4f76-be5c-7c63d7ef8ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:32.076817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3993800-f433-4f06-957c-47b361ecddb9 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:32.092684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ff086331-a172-4f76-be5c-7c63d7ef8ef6 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:32.093892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ff086331-a172-4f76-be5c-7c63d7ef8ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:32.104255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:32.116734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd5c91-8211-40ae-876e-89cd7a3f3a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:32.117900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd5c91-8211-40ae-876e-89cd7a3f3a1f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:32.118716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd5c91-8211-40ae-876e-89cd7a3f3a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:32.118992 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd5c91-8211-40ae-876e-89cd7a3f3a1f None admin] PATH_INFO: `/v3/users/5f9f473391f14769951c95fa506fcefa` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:32.133705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd5c91-8211-40ae-876e-89cd7a3f3a1f None admin] RBAC: Authorizing `identity:get_user(user_id=5f9f473391f14769951c95fa506fcefa)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:32.135352 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd5c91-8211-40ae-876e-89cd7a3f3a1f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:32.137905 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 118/244] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:31 2022] GET /identity/v3/users/5f9f473391f14769951c95fa506fcefa => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:32.139230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a99841-cb90-491b-a183-53171539fa00 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:32.139523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a99841-cb90-491b-a183-53171539fa00 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:32.139802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a99841-cb90-491b-a183-53171539fa00 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:32.141131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 119/245] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:32.145190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a99841-cb90-491b-a183-53171539fa00 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:32.179058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 127/246] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:32.181035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77c0c4-a298-41dd-9a15-07210c96b884 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:32.181511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77c0c4-a298-41dd-9a15-07210c96b884 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:32.181996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77c0c4-a298-41dd-9a15-07210c96b884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:32.200894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-6d77c0c4-a298-41dd-9a15-07210c96b884 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:32.202445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6d77c0c4-a298-41dd-9a15-07210c96b884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:32.277026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:32.283334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a99841-cb90-491b-a183-53171539fa00 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:32.299155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 128/247] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:32.300306 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77c0c4-a298-41dd-9a15-07210c96b884 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:32.340727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f026e-e26f-41d1-b006-6e12e7eda74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:32.341913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f026e-e26f-41d1-b006-6e12e7eda74f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:32.342193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f026e-e26f-41d1-b006-6e12e7eda74f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:32.342468 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f026e-e26f-41d1-b006-6e12e7eda74f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:32.369789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:32.421522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77c0c4-a298-41dd-9a15-07210c96b884 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:32.433244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f026e-e26f-41d1-b006-6e12e7eda74f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:32.435457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f026e-e26f-41d1-b006-6e12e7eda74f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:32.465412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da749583-a36d-4de7-ab93-a33fe544cd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:32.467270 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da749583-a36d-4de7-ab93-a33fe544cd7a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:32.467823 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da749583-a36d-4de7-ab93-a33fe544cd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:32.468129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da749583-a36d-4de7-ab93-a33fe544cd7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:32.483374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da749583-a36d-4de7-ab93-a33fe544cd7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-da749583-a36d-4de7-ab93-a33fe544cd7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:32.501595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:32.502511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:32.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 129/248] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:06:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:32.503957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da749583-a36d-4de7-ab93-a33fe544cd7a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:32.593740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:32.611155 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 120/249] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 07:06:32.613177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f026e-e26f-41d1-b006-6e12e7eda74f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:32.677460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:32.726249 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da749583-a36d-4de7-ab93-a33fe544cd7a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:32.731155 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f026e-e26f-41d1-b006-6e12e7eda74f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:32.782551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160ad66-6780-44b7-bd3b-cd7dea9276c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:32.783235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160ad66-6780-44b7-bd3b-cd7dea9276c6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:32.783523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160ad66-6780-44b7-bd3b-cd7dea9276c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:32.783808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160ad66-6780-44b7-bd3b-cd7dea9276c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:32.784595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160ad66-6780-44b7-bd3b-cd7dea9276c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:32.784909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160ad66-6780-44b7-bd3b-cd7dea9276c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:32.786396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160ad66-6780-44b7-bd3b-cd7dea9276c6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:32.788549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af8d76-2223-4930-8b95-edadcd9508b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:32.789795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8d76-2223-4930-8b95-edadcd9508b4 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:32.790077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8d76-2223-4930-8b95-edadcd9508b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:32.790355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8d76-2223-4930-8b95-edadcd9508b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:32.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 121/250] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:32.796529 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160ad66-6780-44b7-bd3b-cd7dea9276c6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:32.814623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af8d76-2223-4930-8b95-edadcd9508b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-47af8d76-2223-4930-8b95-edadcd9508b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:32.830010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:32.830958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 130/251] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:32.832380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af8d76-2223-4930-8b95-edadcd9508b4 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:32.969809 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af8d76-2223-4930-8b95-edadcd9508b4 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:32.980238 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:33.009620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86941c24-8644-4d33-b881-712c6278b9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:33.010758 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86941c24-8644-4d33-b881-712c6278b9fd None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:33.011056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86941c24-8644-4d33-b881-712c6278b9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:33.011336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86941c24-8644-4d33-b881-712c6278b9fd None admin] PATH_INFO: `/v3/users/ada5f7254b7147fc8c38de6c39727018` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:33.016858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86941c24-8644-4d33-b881-712c6278b9fd None admin] RBAC: Authorizing `identity:get_user(user_id=ada5f7254b7147fc8c38de6c39727018)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:33.018559 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86941c24-8644-4d33-b881-712c6278b9fd None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:33.021286 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 131/252] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:32 2022] GET /identity/v3/users/ada5f7254b7147fc8c38de6c39727018 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:33.023665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86941c24-8644-4d33-b881-712c6278b9fd None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:33.034577 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160ad66-6780-44b7-bd3b-cd7dea9276c6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:33.051996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:33.076708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db0ebf-2973-4bbd-ad6d-b6722e0584d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:33.077580 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db0ebf-2973-4bbd-ad6d-b6722e0584d6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:33.077865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db0ebf-2973-4bbd-ad6d-b6722e0584d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:33.078144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db0ebf-2973-4bbd-ad6d-b6722e0584d6 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:33.080626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db0ebf-2973-4bbd-ad6d-b6722e0584d6 None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:33.081382 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db0ebf-2973-4bbd-ad6d-b6722e0584d6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:33.083582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 122/253] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:32 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:33.084527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db0ebf-2973-4bbd-ad6d-b6722e0584d6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:33.212154 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db0ebf-2973-4bbd-ad6d-b6722e0584d6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:33.242933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86941c24-8644-4d33-b881-712c6278b9fd None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:33.254358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc29f6b-e5dd-4f1f-b41e-d788b4818912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:33.255369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc29f6b-e5dd-4f1f-b41e-d788b4818912 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:33.255658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc29f6b-e5dd-4f1f-b41e-d788b4818912 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:33.255941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc29f6b-e5dd-4f1f-b41e-d788b4818912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:33.283498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc356ba4-5634-481f-8315-f1307ee41859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:33.284448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc356ba4-5634-481f-8315-f1307ee41859 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:33.284933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc356ba4-5634-481f-8315-f1307ee41859 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:33.285319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc356ba4-5634-481f-8315-f1307ee41859 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:33.286446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc356ba4-5634-481f-8315-f1307ee41859 None admin] RBAC: Adding query filter params (name=member) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:33.287011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc356ba4-5634-481f-8315-f1307ee41859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:33.288589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc356ba4-5634-481f-8315-f1307ee41859 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:33.299513 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 132/254] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:33.300483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc356ba4-5634-481f-8315-f1307ee41859 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:33.320661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:33.343074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc29f6b-e5dd-4f1f-b41e-d788b4818912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:33.344576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc29f6b-e5dd-4f1f-b41e-d788b4818912 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:33.347278 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:33.411914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc356ba4-5634-481f-8315-f1307ee41859 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:33.448335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30873064-e626-4a9e-b67b-8ffe4f3c3ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:33.449328 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30873064-e626-4a9e-b67b-8ffe4f3c3ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:33.449762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30873064-e626-4a9e-b67b-8ffe4f3c3ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:33.450737 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30873064-e626-4a9e-b67b-8ffe4f3c3ef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:33.451829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30873064-e626-4a9e-b67b-8ffe4f3c3ef2 None admin] RBAC: Adding query filter params (role.id=b3b7dfbee8c34760b5acbb12576c8fdd, scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d, user.id=5f9f473391f14769951c95fa506fcefa) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:33.452419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30873064-e626-4a9e-b67b-8ffe4f3c3ef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:33.453994 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30873064-e626-4a9e-b67b-8ffe4f3c3ef2 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:33.458909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 123/255] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:33.460171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 133/256] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:33 2022] GET /identity/v3/role_assignments?user.id=5f9f473391f14769951c95fa506fcefa&scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d&role.id=b3b7dfbee8c34760b5acbb12576c8fdd => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:33.460886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e8e10-6179-46d9-bcd0-10da5168fd8a None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:33.461380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e8e10-6179-46d9-bcd0-10da5168fd8a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:33.461800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e8e10-6179-46d9-bcd0-10da5168fd8a None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:33.463732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 124/257] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:33.464811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e8e10-6179-46d9-bcd0-10da5168fd8a None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:33.467104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30873064-e626-4a9e-b67b-8ffe4f3c3ef2 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:33.616634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:33.625162 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e8e10-6179-46d9-bcd0-10da5168fd8a None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:33.676970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e491e-a63d-4396-af52-8323dc24d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:33.678838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e491e-a63d-4396-af52-8323dc24d999 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:33.679391 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e491e-a63d-4396-af52-8323dc24d999 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:33.679841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e491e-a63d-4396-af52-8323dc24d999 None admin] PATH_INFO: `/v3/users/dddd8787eb3643e2a6573146d6d8846f` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:33.681365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e491e-a63d-4396-af52-8323dc24d999 None admin] RBAC: Authorizing `identity:get_user(user_id=dddd8787eb3643e2a6573146d6d8846f)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:33.683202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e491e-a63d-4396-af52-8323dc24d999 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:33.685368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 125/258] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:33 2022] GET /identity/v3/users/dddd8787eb3643e2a6573146d6d8846f => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:33.689615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30873064-e626-4a9e-b67b-8ffe4f3c3ef2 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:33.690153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:33.692574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6509f3-0e2c-4306-9b8a-8dcf6acb67a4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:33.693078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6509f3-0e2c-4306-9b8a-8dcf6acb67a4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:33.693895 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6509f3-0e2c-4306-9b8a-8dcf6acb67a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:33.712407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-aa6509f3-0e2c-4306-9b8a-8dcf6acb67a4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:33.714085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-aa6509f3-0e2c-4306-9b8a-8dcf6acb67a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:33.725585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8d536-8083-4f22-8b07-e20e4ee861b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:33.726604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8d536-8083-4f22-8b07-e20e4ee861b0 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:33.726833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8d536-8083-4f22-8b07-e20e4ee861b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:33.727112 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8d536-8083-4f22-8b07-e20e4ee861b0 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:33.728347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8d536-8083-4f22-8b07-e20e4ee861b0 None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:33.729898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8d536-8083-4f22-8b07-e20e4ee861b0 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:33.732598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 134/259] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:33 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:33.733461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8d536-8083-4f22-8b07-e20e4ee861b0 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:33.791461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 126/260] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:33.793418 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6509f3-0e2c-4306-9b8a-8dcf6acb67a4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:33.921757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8d536-8083-4f22-8b07-e20e4ee861b0 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:33.947515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:33.984427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6509f3-0e2c-4306-9b8a-8dcf6acb67a4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:33.993576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67e72a-e3b7-4ec0-93e6-812e947e792e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:33.994694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67e72a-e3b7-4ec0-93e6-812e947e792e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:33.995174 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67e72a-e3b7-4ec0-93e6-812e947e792e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:33.995642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67e72a-e3b7-4ec0-93e6-812e947e792e None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:33.999853 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67e72a-e3b7-4ec0-93e6-812e947e792e None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:34.001556 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67e72a-e3b7-4ec0-93e6-812e947e792e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:34.003491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 135/261] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:33 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:34.006764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67e72a-e3b7-4ec0-93e6-812e947e792e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:34.053994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608460a-5ca1-4c29-8949-0549d0d56525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:34.054669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608460a-5ca1-4c29-8949-0549d0d56525 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:34.054956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608460a-5ca1-4c29-8949-0549d0d56525 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:34.055232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608460a-5ca1-4c29-8949-0549d0d56525 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:34.056105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608460a-5ca1-4c29-8949-0549d0d56525 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d, user.id=ada5f7254b7147fc8c38de6c39727018) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:34.056410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608460a-5ca1-4c29-8949-0549d0d56525 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:34.057858 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608460a-5ca1-4c29-8949-0549d0d56525 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:34.064544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 127/262] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:33 2022] GET /identity/v3/role_assignments?user.id=ada5f7254b7147fc8c38de6c39727018&scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:34.065354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608460a-5ca1-4c29-8949-0549d0d56525 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:34.067472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:34.136772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67e72a-e3b7-4ec0-93e6-812e947e792e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:34.181348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ddaec-ca91-437a-98bb-bac1c982da93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:34.182539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ddaec-ca91-437a-98bb-bac1c982da93 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:34.183026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ddaec-ca91-437a-98bb-bac1c982da93 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:34.183517 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ddaec-ca91-437a-98bb-bac1c982da93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:34.197127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ddaec-ca91-437a-98bb-bac1c982da93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-6c5ddaec-ca91-437a-98bb-bac1c982da93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:34.214486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:34.215447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:34.216423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:34.216423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:34.216423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:34.216423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:34.216423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:34.216423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:34.216423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:34.216423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:34.216423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:34.216423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:34.216423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:34.216423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:34.219064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 136/263] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:06:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:34.225040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ddaec-ca91-437a-98bb-bac1c982da93 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:34.361300 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608460a-5ca1-4c29-8949-0549d0d56525 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:34.395757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:34.401449 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ddaec-ca91-437a-98bb-bac1c982da93 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:34.421348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b7ce9-0517-43ac-8636-57de51ee94b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:34.422260 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b7ce9-0517-43ac-8636-57de51ee94b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:34.422545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b7ce9-0517-43ac-8636-57de51ee94b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:34.422826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b7ce9-0517-43ac-8636-57de51ee94b5 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d/users/dddd8787eb3643e2a6573146d6d8846f/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:34.429518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b7ce9-0517-43ac-8636-57de51ee94b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=36fac836e8de4d52aa468bc5deaf2a2d, user_id=dddd8787eb3643e2a6573146d6d8846f, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:34.431005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b7ce9-0517-43ac-8636-57de51ee94b5 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:34.455890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a60d14-385a-436a-8d09-cb694b513e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:34.456568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a60d14-385a-436a-8d09-cb694b513e93 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:34.457345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a60d14-385a-436a-8d09-cb694b513e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:34.457939 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a60d14-385a-436a-8d09-cb694b513e93 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:34.458901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a60d14-385a-436a-8d09-cb694b513e93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:34.459437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a60d14-385a-436a-8d09-cb694b513e93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:34.461145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a60d14-385a-436a-8d09-cb694b513e93 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:34.462889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 128/264] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:06:34 2022] PUT /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d/users/dddd8787eb3643e2a6573146d6d8846f/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:34.471784 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 137/265] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:06:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:34.477351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b7ce9-0517-43ac-8636-57de51ee94b5 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:34.484665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:34.590563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b7ce9-0517-43ac-8636-57de51ee94b5 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:34.628478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbea22-6782-402a-be45-5659f5b93506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:34.629758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbea22-6782-402a-be45-5659f5b93506 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:34.630242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbea22-6782-402a-be45-5659f5b93506 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:34.630713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbea22-6782-402a-be45-5659f5b93506 None admin] PATH_INFO: `/v3/users/ada5f7254b7147fc8c38de6c39727018` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:34.632137 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbea22-6782-402a-be45-5659f5b93506 None admin] RBAC: Authorizing `identity:get_user(user_id=ada5f7254b7147fc8c38de6c39727018)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:34.633840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbea22-6782-402a-be45-5659f5b93506 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:34.635976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 129/266] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:34 2022] GET /identity/v3/users/ada5f7254b7147fc8c38de6c39727018 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:34.642361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a60d14-385a-436a-8d09-cb694b513e93 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:34.760698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a60d14-385a-436a-8d09-cb694b513e93 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:34.780393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b2773-ecf3-4708-8b3c-7f34996d21ec None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:34.781020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b2773-ecf3-4708-8b3c-7f34996d21ec None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:34.781806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b2773-ecf3-4708-8b3c-7f34996d21ec None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:34.783633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 130/267] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:34.791927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067943b3-a65f-42f3-8c04-b26f248594e6 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:34.792584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067943b3-a65f-42f3-8c04-b26f248594e6 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:34.793196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067943b3-a65f-42f3-8c04-b26f248594e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:34.801731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cae3c-523a-4a62-ad67-969711b1f133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:34.802392 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cae3c-523a-4a62-ad67-969711b1f133 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:34.802679 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cae3c-523a-4a62-ad67-969711b1f133 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:34.802958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cae3c-523a-4a62-ad67-969711b1f133 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:34.813820 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-067943b3-a65f-42f3-8c04-b26f248594e6 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:34.815554 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-067943b3-a65f-42f3-8c04-b26f248594e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:34.837045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:34.881044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cae3c-523a-4a62-ad67-969711b1f133 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:34.882520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cae3c-523a-4a62-ad67-969711b1f133 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:34.898214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 131/268] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:34.913002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecabe4-9cf4-4e7d-92e5-83fccebc3e1d None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:34.913600 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecabe4-9cf4-4e7d-92e5-83fccebc3e1d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:34.914143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecabe4-9cf4-4e7d-92e5-83fccebc3e1d None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:34.916114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 132/269] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:34.921293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e60f83-16b8-422b-aafc-805dca21f26b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:34.921887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e60f83-16b8-422b-aafc-805dca21f26b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:34.922373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e60f83-16b8-422b-aafc-805dca21f26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:34.929552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:34.947109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-28e60f83-16b8-422b-aafc-805dca21f26b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:34.948537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-28e60f83-16b8-422b-aafc-805dca21f26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:34.991740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 138/270] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:34.992927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cae3c-523a-4a62-ad67-969711b1f133 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:35.043691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 133/271] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:35.045170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e60f83-16b8-422b-aafc-805dca21f26b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:35.059144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:35.113336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cae3c-523a-4a62-ad67-969711b1f133 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:35.157519 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c64e42-f296-4c60-9ece-83311d824d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:35.158717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c64e42-f296-4c60-9ece-83311d824d6e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:35.159253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c64e42-f296-4c60-9ece-83311d824d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:35.159723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c64e42-f296-4c60-9ece-83311d824d6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:35.161178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c64e42-f296-4c60-9ece-83311d824d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:35.162833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c64e42-f296-4c60-9ece-83311d824d6e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:35.165054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 139/272] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:35.166161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c64e42-f296-4c60-9ece-83311d824d6e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:35.181219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e60f83-16b8-422b-aafc-805dca21f26b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:35.221777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:35.230064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745d457-8191-4bd5-84e0-6485336d28e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:35.231409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745d457-8191-4bd5-84e0-6485336d28e6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:35.232432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745d457-8191-4bd5-84e0-6485336d28e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:35.233276 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745d457-8191-4bd5-84e0-6485336d28e6 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:35.235019 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745d457-8191-4bd5-84e0-6485336d28e6 None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:35.237551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745d457-8191-4bd5-84e0-6485336d28e6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:35.239437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 134/273] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:35 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:35.244578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745d457-8191-4bd5-84e0-6485336d28e6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:35.290573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c64e42-f296-4c60-9ece-83311d824d6e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:35.321253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:35.334887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92744b09-fd90-48e3-b69a-441d5c59f9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:35.336001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92744b09-fd90-48e3-b69a-441d5c59f9dc None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:35.336541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92744b09-fd90-48e3-b69a-441d5c59f9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:35.337022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92744b09-fd90-48e3-b69a-441d5c59f9dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:35.352673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92744b09-fd90-48e3-b69a-441d5c59f9dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:35.364721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745d457-8191-4bd5-84e0-6485336d28e6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-92744b09-fd90-48e3-b69a-441d5c59f9dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:35.365404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:35.366254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:35.367043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:35.367043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:35.367043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:35.367043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:35.367043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:35.367043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:35.367043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:35.367043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:35.367043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:35.367043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:35.367043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:35.367043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:35.369536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 140/274] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:35.372852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92744b09-fd90-48e3-b69a-441d5c59f9dc None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:35.417142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7d596-8101-498d-ab08-b6d7f166eac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:35.417809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7d596-8101-498d-ab08-b6d7f166eac7 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:35.418090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7d596-8101-498d-ab08-b6d7f166eac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:35.418361 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7d596-8101-498d-ab08-b6d7f166eac7 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:35.419133 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7d596-8101-498d-ab08-b6d7f166eac7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:35.421217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7d596-8101-498d-ab08-b6d7f166eac7 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:35.446325 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 135/275] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:06:35 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:35.447615 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7d596-8101-498d-ab08-b6d7f166eac7 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:35.510364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92744b09-fd90-48e3-b69a-441d5c59f9dc None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:35.516793 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:35.556701 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711597c-f961-4229-906c-2d1a4b6cc6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:35.557864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711597c-f961-4229-906c-2d1a4b6cc6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:35.558166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711597c-f961-4229-906c-2d1a4b6cc6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:35.558461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711597c-f961-4229-906c-2d1a4b6cc6e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:35.559322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711597c-f961-4229-906c-2d1a4b6cc6e2 None admin] RBAC: Adding query filter params (role.id=22d7b181c7e041329c59c3eb6b492cd7, scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d, user.id=ada5f7254b7147fc8c38de6c39727018) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:35.559676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711597c-f961-4229-906c-2d1a4b6cc6e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:35.561579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711597c-f961-4229-906c-2d1a4b6cc6e2 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:35.571420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 141/276] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:35 2022] GET /identity/v3/role_assignments?user.id=ada5f7254b7147fc8c38de6c39727018&scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d&role.id=22d7b181c7e041329c59c3eb6b492cd7 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:35.576108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7d596-8101-498d-ab08-b6d7f166eac7 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:35.614953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395e126-3aa6-4a47-b2e1-e831fdfacd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:35.615653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395e126-3aa6-4a47-b2e1-e831fdfacd84 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:35.615936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395e126-3aa6-4a47-b2e1-e831fdfacd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:35.616252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395e126-3aa6-4a47-b2e1-e831fdfacd84 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:35.617018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395e126-3aa6-4a47-b2e1-e831fdfacd84 None admin] RBAC: Adding query filter params (name=member) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:35.617326 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395e126-3aa6-4a47-b2e1-e831fdfacd84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:35.618761 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395e126-3aa6-4a47-b2e1-e831fdfacd84 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:35.632604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 136/277] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:35.634442 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711597c-f961-4229-906c-2d1a4b6cc6e2 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:35.720128 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:35.777609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711597c-f961-4229-906c-2d1a4b6cc6e2 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:35.822228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1062a-9a0d-4603-85c9-33525d0e8eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:35.823039 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1062a-9a0d-4603-85c9-33525d0e8eea None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:35.823317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1062a-9a0d-4603-85c9-33525d0e8eea None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:35.823590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1062a-9a0d-4603-85c9-33525d0e8eea None admin] PATH_INFO: `/v3/users/37730a8df26140599c95f5fc74ff0f08` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:35.843110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1062a-9a0d-4603-85c9-33525d0e8eea None admin] RBAC: Authorizing `identity:get_user(user_id=37730a8df26140599c95f5fc74ff0f08)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:35.844629 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1062a-9a0d-4603-85c9-33525d0e8eea None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:35.847114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 142/278] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:35 2022] GET /identity/v3/users/37730a8df26140599c95f5fc74ff0f08 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:35.853420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395e126-3aa6-4a47-b2e1-e831fdfacd84 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:35.994090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395e126-3aa6-4a47-b2e1-e831fdfacd84 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:36.015691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:36.038038 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6cd87-bc9f-4634-8700-2e0e0188f455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:36.038717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6cd87-bc9f-4634-8700-2e0e0188f455 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:36.039004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6cd87-bc9f-4634-8700-2e0e0188f455 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:36.039285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6cd87-bc9f-4634-8700-2e0e0188f455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:36.040167 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6cd87-bc9f-4634-8700-2e0e0188f455 None admin] RBAC: Adding query filter params (role.id=ce36ba2b01744465803d0533b2a47592, user.id=37730a8df26140599c95f5fc74ff0f08, scope.system=all) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:36.040485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6cd87-bc9f-4634-8700-2e0e0188f455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:36.041964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6cd87-bc9f-4634-8700-2e0e0188f455 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:36.048358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 137/279] 10.222.0.33 () {56 vars in 1308 bytes} [Fri Jun 17 07:06:35 2022] GET /identity/v3/role_assignments?user.id=37730a8df26140599c95f5fc74ff0f08&scope.system=all&role.id=ce36ba2b01744465803d0533b2a47592 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:36.078569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db4695-8ea8-4b1a-b515-3e125e53c0b4 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:36.078775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db4695-8ea8-4b1a-b515-3e125e53c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:36.079058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db4695-8ea8-4b1a-b515-3e125e53c0b4 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:36.080832 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 143/280] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:36.087735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9faf8-96a9-4ca8-a36e-de1eb8725461 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:36.088093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9faf8-96a9-4ca8-a36e-de1eb8725461 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:36.088384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9faf8-96a9-4ca8-a36e-de1eb8725461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:36.107165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-63a9faf8-96a9-4ca8-a36e-de1eb8725461 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:36.108439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-63a9faf8-96a9-4ca8-a36e-de1eb8725461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:36.183039 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 138/281] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:36.206286 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db4695-8ea8-4b1a-b515-3e125e53c0b4 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:36.313583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db4695-8ea8-4b1a-b515-3e125e53c0b4 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:36.325854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:36.351079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c23a48-cb79-428d-9e26-b8330cc18db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:36.351800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c23a48-cb79-428d-9e26-b8330cc18db8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:36.352178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c23a48-cb79-428d-9e26-b8330cc18db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:36.352458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c23a48-cb79-428d-9e26-b8330cc18db8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:36.363836 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c23a48-cb79-428d-9e26-b8330cc18db8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-65c23a48-cb79-428d-9e26-b8330cc18db8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:36.371770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:36.372742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:36.373525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 144/282] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:36.379343 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9faf8-96a9-4ca8-a36e-de1eb8725461 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:36.481817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9faf8-96a9-4ca8-a36e-de1eb8725461 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:36.495320 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:36.520811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a41215-7185-46eb-8d5a-d4cdd8b1262d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:36.521497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a41215-7185-46eb-8d5a-d4cdd8b1262d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:36.521779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a41215-7185-46eb-8d5a-d4cdd8b1262d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:36.522055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a41215-7185-46eb-8d5a-d4cdd8b1262d None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:36.522802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a41215-7185-46eb-8d5a-d4cdd8b1262d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:36.523107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a41215-7185-46eb-8d5a-d4cdd8b1262d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:36.524515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a41215-7185-46eb-8d5a-d4cdd8b1262d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:36.535300 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 139/283] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:36.540702 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c23a48-cb79-428d-9e26-b8330cc18db8 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:36.642940 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c23a48-cb79-428d-9e26-b8330cc18db8 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:36.677437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080feb9f-b137-4917-8c26-40443d0800d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:36.678366 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080feb9f-b137-4917-8c26-40443d0800d8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:36.678743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080feb9f-b137-4917-8c26-40443d0800d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:36.679022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080feb9f-b137-4917-8c26-40443d0800d8 None admin] PATH_INFO: `/v3/users/5f9f473391f14769951c95fa506fcefa` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:36.680238 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080feb9f-b137-4917-8c26-40443d0800d8 None admin] RBAC: Authorizing `identity:get_user(user_id=5f9f473391f14769951c95fa506fcefa)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:36.681682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080feb9f-b137-4917-8c26-40443d0800d8 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:36.683641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 145/284] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:36 2022] GET /identity/v3/users/5f9f473391f14769951c95fa506fcefa => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:36.689613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a41215-7185-46eb-8d5a-d4cdd8b1262d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:36.792651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a41215-7185-46eb-8d5a-d4cdd8b1262d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:36.827758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6fefd-cdab-4baa-aa5b-9a0bf0d51243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:36.828676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6fefd-cdab-4baa-aa5b-9a0bf0d51243 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:36.829052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6fefd-cdab-4baa-aa5b-9a0bf0d51243 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:36.829327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6fefd-cdab-4baa-aa5b-9a0bf0d51243 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:36.830594 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6fefd-cdab-4baa-aa5b-9a0bf0d51243 None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:36.832100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6fefd-cdab-4baa-aa5b-9a0bf0d51243 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:36.833825 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 140/285] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:36 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:36.839594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080feb9f-b137-4917-8c26-40443d0800d8 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:36.881897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:36.943299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080feb9f-b137-4917-8c26-40443d0800d8 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:36.978437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8d821-9052-4bb0-bad7-fd85519b6784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:36.979333 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d821-9052-4bb0-bad7-fd85519b6784 None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:36.979614 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d821-9052-4bb0-bad7-fd85519b6784 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:36.979891 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d821-9052-4bb0-bad7-fd85519b6784 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d/users/5f9f473391f14769951c95fa506fcefa/roles/b3b7dfbee8c34760b5acbb12576c8fdd` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:36.982445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d821-9052-4bb0-bad7-fd85519b6784 None admin] RBAC: Authorizing `identity:create_grant(project_id=36fac836e8de4d52aa468bc5deaf2a2d, user_id=5f9f473391f14769951c95fa506fcefa, role_id=b3b7dfbee8c34760b5acbb12576c8fdd)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:36.983934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d821-9052-4bb0-bad7-fd85519b6784 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:37.000613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 146/286] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:06:36 2022] PUT /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d/users/5f9f473391f14769951c95fa506fcefa/roles/b3b7dfbee8c34760b5acbb12576c8fdd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:37.057770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b839629-7341-4893-89f0-919687a5759b None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:37.058100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b839629-7341-4893-89f0-919687a5759b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:37.058330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b839629-7341-4893-89f0-919687a5759b None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:37.060450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 141/287] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:37.065989 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603f18d-5e66-4a82-9197-89e8c275dd23 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:37.066291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603f18d-5e66-4a82-9197-89e8c275dd23 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:37.066600 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603f18d-5e66-4a82-9197-89e8c275dd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:37.091280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-b603f18d-5e66-4a82-9197-89e8c275dd23 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:37.092617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b603f18d-5e66-4a82-9197-89e8c275dd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:37.168368 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 147/288] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:37.188541 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b839629-7341-4893-89f0-919687a5759b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:37.240087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:37.306519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b839629-7341-4893-89f0-919687a5759b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:37.341994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeaee3-d518-4e4c-a845-4ab2a2a76f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:37.342798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeaee3-d518-4e4c-a845-4ab2a2a76f6c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:37.343087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeaee3-d518-4e4c-a845-4ab2a2a76f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:37.343366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeaee3-d518-4e4c-a845-4ab2a2a76f6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:37.353899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eeaee3-d518-4e4c-a845-4ab2a2a76f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-03eeaee3-d518-4e4c-a845-4ab2a2a76f6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:37.362235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:37.363113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:37.364051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 142/289] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:37.369200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603f18d-5e66-4a82-9197-89e8c275dd23 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:37.410108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:37.485981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603f18d-5e66-4a82-9197-89e8c275dd23 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:37.534218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71607132-ba6f-4f3d-a680-a12a70de8058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:37.534814 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71607132-ba6f-4f3d-a680-a12a70de8058 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:37.535100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71607132-ba6f-4f3d-a680-a12a70de8058 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:37.535378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71607132-ba6f-4f3d-a680-a12a70de8058 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:37.536158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71607132-ba6f-4f3d-a680-a12a70de8058 None admin] RBAC: Adding query filter params (name=member) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:37.536471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71607132-ba6f-4f3d-a680-a12a70de8058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:37.537848 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71607132-ba6f-4f3d-a680-a12a70de8058 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:37.549054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 148/290] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:37.554220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeaee3-d518-4e4c-a845-4ab2a2a76f6c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:37.659944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeaee3-d518-4e4c-a845-4ab2a2a76f6c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:37.661538 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:37.695393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35f938-56ce-4603-beb8-e1c49050698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:37.696304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35f938-56ce-4603-beb8-e1c49050698b None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:37.696588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35f938-56ce-4603-beb8-e1c49050698b None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:37.696864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35f938-56ce-4603-beb8-e1c49050698b None admin] PATH_INFO: `/v3/users/dddd8787eb3643e2a6573146d6d8846f` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:37.698080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35f938-56ce-4603-beb8-e1c49050698b None admin] RBAC: Authorizing `identity:get_user(user_id=dddd8787eb3643e2a6573146d6d8846f)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:37.699543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35f938-56ce-4603-beb8-e1c49050698b None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:37.702049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 143/291] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:37 2022] GET /identity/v3/users/dddd8787eb3643e2a6573146d6d8846f => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:37.707066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71607132-ba6f-4f3d-a680-a12a70de8058 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:37.800143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:37.809334 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71607132-ba6f-4f3d-a680-a12a70de8058 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:37.852572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ac5c6-1b96-40ba-952e-1d37672ae3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:37.853227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ac5c6-1b96-40ba-952e-1d37672ae3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:37.853514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ac5c6-1b96-40ba-952e-1d37672ae3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:37.853792 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ac5c6-1b96-40ba-952e-1d37672ae3c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:37.947739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ac5c6-1b96-40ba-952e-1d37672ae3c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:37.949548 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ac5c6-1b96-40ba-952e-1d37672ae3c3 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:38.072052 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:38.089056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 149/292] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:38.094291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35f938-56ce-4603-beb8-e1c49050698b None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:38.239807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35f938-56ce-4603-beb8-e1c49050698b None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:38.283016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec571f09-5627-43b6-a4ec-03c46ec8343d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:38.283881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec571f09-5627-43b6-a4ec-03c46ec8343d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:38.284192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec571f09-5627-43b6-a4ec-03c46ec8343d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:38.284470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec571f09-5627-43b6-a4ec-03c46ec8343d None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:38.288792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec571f09-5627-43b6-a4ec-03c46ec8343d None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:38.290244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec571f09-5627-43b6-a4ec-03c46ec8343d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:38.292284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 144/293] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:38 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:38.297939 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ac5c6-1b96-40ba-952e-1d37672ae3c3 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:38.437965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd2149-dd0d-4620-bd90-2f7566a93145 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:38.437965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd2149-dd0d-4620-bd90-2f7566a93145 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:38.437965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd2149-dd0d-4620-bd90-2f7566a93145 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:38.437965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 145/294] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:38.442921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11964da9-60d7-41dd-8304-600ab13d1561 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:38.443212 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11964da9-60d7-41dd-8304-600ab13d1561 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:38.443496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11964da9-60d7-41dd-8304-600ab13d1561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:38.462823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-11964da9-60d7-41dd-8304-600ab13d1561 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:38.464016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-11964da9-60d7-41dd-8304-600ab13d1561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:38.487914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ac5c6-1b96-40ba-952e-1d37672ae3c3 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:38.501825 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:38.543938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046fd78-f0f4-43ce-96ed-0dcab290a35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:38.545541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046fd78-f0f4-43ce-96ed-0dcab290a35c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:38.546191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046fd78-f0f4-43ce-96ed-0dcab290a35c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:38.547668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046fd78-f0f4-43ce-96ed-0dcab290a35c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:38.548752 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046fd78-f0f4-43ce-96ed-0dcab290a35c None admin] RBAC: Adding query filter params (role.id=ce36ba2b01744465803d0533b2a47592, scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d, user.id=dddd8787eb3643e2a6573146d6d8846f) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:38.549460 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046fd78-f0f4-43ce-96ed-0dcab290a35c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:38.552216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046fd78-f0f4-43ce-96ed-0dcab290a35c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:38.561764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 150/295] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:38 2022] GET /identity/v3/role_assignments?user.id=dddd8787eb3643e2a6573146d6d8846f&scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d&role.id=ce36ba2b01744465803d0533b2a47592 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:38.569840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 146/296] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:38.605964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046fd78-f0f4-43ce-96ed-0dcab290a35c None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:38.686815 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:38.740960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046fd78-f0f4-43ce-96ed-0dcab290a35c None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:38.775723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1c1a3-6473-4095-a340-3dd28cfd0f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:38.776837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1c1a3-6473-4095-a340-3dd28cfd0f45 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:38.777123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1c1a3-6473-4095-a340-3dd28cfd0f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:38.777401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1c1a3-6473-4095-a340-3dd28cfd0f45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:38.778617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1c1a3-6473-4095-a340-3dd28cfd0f45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:38.780590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1c1a3-6473-4095-a340-3dd28cfd0f45 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:38.782497 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 151/297] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:38.787655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11964da9-60d7-41dd-8304-600ab13d1561 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:38.905534 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11964da9-60d7-41dd-8304-600ab13d1561 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:38.939879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28849d74-6b81-4089-a273-8fea8fa2ff96 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:38.940180 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28849d74-6b81-4089-a273-8fea8fa2ff96 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:38.940393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28849d74-6b81-4089-a273-8fea8fa2ff96 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:38.942009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 152/298] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:38.942498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd49d1e-9b3c-4b7b-8aba-f555cb3dd4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:38.943571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd49d1e-9b3c-4b7b-8aba-f555cb3dd4e1 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:38.944220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd49d1e-9b3c-4b7b-8aba-f555cb3dd4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:38.944698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd49d1e-9b3c-4b7b-8aba-f555cb3dd4e1 None admin] PATH_INFO: `/v3/groups` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:38.945642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd49d1e-9b3c-4b7b-8aba-f555cb3dd4e1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:38.946659 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec60d1d-8ec3-41ab-9ee0-71bbbf4b0569 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:38.946950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec60d1d-8ec3-41ab-9ee0-71bbbf4b0569 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:38.947228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec60d1d-8ec3-41ab-9ee0-71bbbf4b0569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:38.947961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd49d1e-9b3c-4b7b-8aba-f555cb3dd4e1 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:38.971146 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-dec60d1d-8ec3-41ab-9ee0-71bbbf4b0569 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:38.971630 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 147/299] 10.222.0.33 () {60 vars in 1134 bytes} [Fri Jun 17 07:06:38 2022] POST /identity/v3/groups => generated 229 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:38.972382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-dec60d1d-8ec3-41ab-9ee0-71bbbf4b0569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:39.055085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe3126-a5cf-4312-b4eb-4286de751280 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:39.055785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe3126-a5cf-4312-b4eb-4286de751280 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:39.056407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe3126-a5cf-4312-b4eb-4286de751280 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:39.058846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 148/300] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:06:39.064446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f784bf5-369c-498a-8533-bc543a22d0be None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:39.065050 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f784bf5-369c-498a-8533-bc543a22d0be None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:39.065534 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f784bf5-369c-498a-8533-bc543a22d0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:39.068055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 153/301] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:39.085731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-7f784bf5-369c-498a-8533-bc543a22d0be None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:39.087377 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7f784bf5-369c-498a-8533-bc543a22d0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:39.093826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec60d1d-8ec3-41ab-9ee0-71bbbf4b0569 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:39.121569 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:39.169804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:39.188801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 149/302] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:39.212055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f784bf5-369c-498a-8533-bc543a22d0be None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:39.225663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec60d1d-8ec3-41ab-9ee0-71bbbf4b0569 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:39.274729 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7465e5-d81d-4aa2-b0e6-354db4e57a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:39.276083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7465e5-d81d-4aa2-b0e6-354db4e57a0a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:39.276730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7465e5-d81d-4aa2-b0e6-354db4e57a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:39.277103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7465e5-d81d-4aa2-b0e6-354db4e57a0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:39.291287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7465e5-d81d-4aa2-b0e6-354db4e57a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-dd7465e5-d81d-4aa2-b0e6-354db4e57a0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:39.302347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:39.303419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:39.304430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:39.304430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:39.304430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:39.304430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:39.304430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:39.304430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:39.304430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:39.304430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:39.304430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:39.304430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:39.304430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:39.304430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:39.309447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 154/303] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:06:39.315250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7465e5-d81d-4aa2-b0e6-354db4e57a0a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:39.346445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f784bf5-369c-498a-8533-bc543a22d0be None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:39.385276 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39660cf-d8fc-4bce-bcde-ac640c75f394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:39.386178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39660cf-d8fc-4bce-bcde-ac640c75f394 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:39.386528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39660cf-d8fc-4bce-bcde-ac640c75f394 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:39.386906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39660cf-d8fc-4bce-bcde-ac640c75f394 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:39.399041 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39660cf-d8fc-4bce-bcde-ac640c75f394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-d39660cf-d8fc-4bce-bcde-ac640c75f394 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:39.407296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:39.408250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:39.409051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 150/304] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:06:39.414225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39660cf-d8fc-4bce-bcde-ac640c75f394 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:39.435038 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:39.446219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7465e5-d81d-4aa2-b0e6-354db4e57a0a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:39.462843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:39.485593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a46af9-c336-4ac6-a147-9c3c8c02fbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:39.486396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a46af9-c336-4ac6-a147-9c3c8c02fbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:39.486698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a46af9-c336-4ac6-a147-9c3c8c02fbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:39.487125 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a46af9-c336-4ac6-a147-9c3c8c02fbb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:39.487877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a46af9-c336-4ac6-a147-9c3c8c02fbb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:39.488428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a46af9-c336-4ac6-a147-9c3c8c02fbb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:39.489758 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a46af9-c336-4ac6-a147-9c3c8c02fbb3 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:39.503215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 155/305] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:39.504278 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a46af9-c336-4ac6-a147-9c3c8c02fbb3 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:39.534616 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39660cf-d8fc-4bce-bcde-ac640c75f394 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:39.579382 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64fce3-726b-4bde-b9ac-5b4a7e77baee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:39.580790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64fce3-726b-4bde-b9ac-5b4a7e77baee None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:39.581305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64fce3-726b-4bde-b9ac-5b4a7e77baee None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:39.581726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64fce3-726b-4bde-b9ac-5b4a7e77baee None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:39.582620 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64fce3-726b-4bde-b9ac-5b4a7e77baee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:39.583593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64fce3-726b-4bde-b9ac-5b4a7e77baee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:39.585648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64fce3-726b-4bde-b9ac-5b4a7e77baee None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:39.597913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 151/306] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:39.603458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64fce3-726b-4bde-b9ac-5b4a7e77baee None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:39.615362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a46af9-c336-4ac6-a147-9c3c8c02fbb3 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:39.654130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c36283-504b-4bfa-8a3b-498c81907c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:39.655327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c36283-504b-4bfa-8a3b-498c81907c36 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:39.655669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c36283-504b-4bfa-8a3b-498c81907c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:39.656008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c36283-504b-4bfa-8a3b-498c81907c36 None admin] PATH_INFO: `/v3/users/37730a8df26140599c95f5fc74ff0f08` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:39.657546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c36283-504b-4bfa-8a3b-498c81907c36 None admin] RBAC: Authorizing `identity:get_user(user_id=37730a8df26140599c95f5fc74ff0f08)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:39.659191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c36283-504b-4bfa-8a3b-498c81907c36 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:39.661037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 156/307] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:39 2022] GET /identity/v3/users/37730a8df26140599c95f5fc74ff0f08 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:39.666595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c36283-504b-4bfa-8a3b-498c81907c36 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:39.707258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:39.711219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:39.731922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64fce3-726b-4bde-b9ac-5b4a7e77baee None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:39.768878 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f73855-0c91-4ddd-b3b8-e36d29a71009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:39.769984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f73855-0c91-4ddd-b3b8-e36d29a71009 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:39.770440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f73855-0c91-4ddd-b3b8-e36d29a71009 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:39.770561 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c36283-504b-4bfa-8a3b-498c81907c36 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:39.770966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f73855-0c91-4ddd-b3b8-e36d29a71009 None admin] PATH_INFO: `/v3/users/5ba45f1acde54b03b9054267da6fd632` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:39.785988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f73855-0c91-4ddd-b3b8-e36d29a71009 None admin] RBAC: Authorizing `identity:get_user(user_id=5ba45f1acde54b03b9054267da6fd632)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:39.787471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f73855-0c91-4ddd-b3b8-e36d29a71009 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:39.789503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 152/308] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:39 2022] GET /identity/v3/users/5ba45f1acde54b03b9054267da6fd632 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:39.790842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e0f55-ae31-4bd0-b0e0-f34c807f16ab None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:39.791273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e0f55-ae31-4bd0-b0e0-f34c807f16ab None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:39.791690 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e0f55-ae31-4bd0-b0e0-f34c807f16ab None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:39.792950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 153/309] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:39.794596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e0f55-ae31-4bd0-b0e0-f34c807f16ab None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:39.811401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af21433-6a75-4d01-9241-a39370f00bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:39.812304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af21433-6a75-4d01-9241-a39370f00bd6 None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:39.812591 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af21433-6a75-4d01-9241-a39370f00bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:39.812870 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af21433-6a75-4d01-9241-a39370f00bd6 None admin] PATH_INFO: `/v3/system/users/37730a8df26140599c95f5fc74ff0f08/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:39.814340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af21433-6a75-4d01-9241-a39370f00bd6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=37730a8df26140599c95f5fc74ff0f08, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:39.815708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af21433-6a75-4d01-9241-a39370f00bd6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:39.825837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 157/310] 10.222.0.33 () {58 vars in 1334 bytes} [Fri Jun 17 07:06:39 2022] PUT /identity/v3/system/users/37730a8df26140599c95f5fc74ff0f08/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:39.827075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f3c45-dcd2-47da-b218-3cc2c5dd91e9 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:39.827364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f3c45-dcd2-47da-b218-3cc2c5dd91e9 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:39.827644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f3c45-dcd2-47da-b218-3cc2c5dd91e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:39.850754 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-117f3c45-dcd2-47da-b218-3cc2c5dd91e9 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:39.852346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-117f3c45-dcd2-47da-b218-3cc2c5dd91e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:39.923453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e0f55-ae31-4bd0-b0e0-f34c807f16ab None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:39.931741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 158/311] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:39.955046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f3c45-dcd2-47da-b218-3cc2c5dd91e9 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:39.973488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbcfa6-b6a4-4951-9554-08acb9ccbc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:39.974544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbcfa6-b6a4-4951-9554-08acb9ccbc72 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:39.975108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbcfa6-b6a4-4951-9554-08acb9ccbc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:39.975585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbcfa6-b6a4-4951-9554-08acb9ccbc72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:39.976708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbcfa6-b6a4-4951-9554-08acb9ccbc72 None admin] RBAC: Adding query filter params (role.id=b3b7dfbee8c34760b5acbb12576c8fdd, user.id=5ba45f1acde54b03b9054267da6fd632, scope.system=all) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:39.977204 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbcfa6-b6a4-4951-9554-08acb9ccbc72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:39.979033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbcfa6-b6a4-4951-9554-08acb9ccbc72 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:39.986383 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 154/312] 10.222.0.33 () {56 vars in 1308 bytes} [Fri Jun 17 07:06:39 2022] GET /identity/v3/role_assignments?user.id=5ba45f1acde54b03b9054267da6fd632&scope.system=all&role.id=b3b7dfbee8c34760b5acbb12576c8fdd => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:40.000969 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:40.081344 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f3c45-dcd2-47da-b218-3cc2c5dd91e9 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:40.119522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a0db8-a626-42b5-8994-762cb4f0cf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:40.120388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a0db8-a626-42b5-8994-762cb4f0cf97 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:40.120641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a0db8-a626-42b5-8994-762cb4f0cf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:40.120922 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a0db8-a626-42b5-8994-762cb4f0cf97 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:40.133379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a0db8-a626-42b5-8994-762cb4f0cf97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-131a0db8-a626-42b5-8994-762cb4f0cf97 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:40.141584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:40.142505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:40.143317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 159/313] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:06:40.148895 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbcfa6-b6a4-4951-9554-08acb9ccbc72 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:40.152969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:40.259062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbcfa6-b6a4-4951-9554-08acb9ccbc72 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:40.294979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6252cd-4a29-4293-9ff3-fcbdebf33f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:40.295651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6252cd-4a29-4293-9ff3-fcbdebf33f4c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:40.295943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6252cd-4a29-4293-9ff3-fcbdebf33f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:40.296255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6252cd-4a29-4293-9ff3-fcbdebf33f4c None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:40.297002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6252cd-4a29-4293-9ff3-fcbdebf33f4c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:40.297314 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6252cd-4a29-4293-9ff3-fcbdebf33f4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:40.298750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6252cd-4a29-4293-9ff3-fcbdebf33f4c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:40.307435 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 155/314] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:40.312967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a0db8-a626-42b5-8994-762cb4f0cf97 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:40.414612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:40.421082 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a0db8-a626-42b5-8994-762cb4f0cf97 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:40.455101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445f98c-9c2e-42f5-afc7-22af34b9735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:40.456031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445f98c-9c2e-42f5-afc7-22af34b9735f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:40.456342 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445f98c-9c2e-42f5-afc7-22af34b9735f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:40.456662 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445f98c-9c2e-42f5-afc7-22af34b9735f None admin] PATH_INFO: `/v3/users/5f9f473391f14769951c95fa506fcefa` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:40.457944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445f98c-9c2e-42f5-afc7-22af34b9735f None admin] RBAC: Authorizing `identity:get_user(user_id=5f9f473391f14769951c95fa506fcefa)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:40.459390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445f98c-9c2e-42f5-afc7-22af34b9735f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:40.461358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 160/315] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:40 2022] GET /identity/v3/users/5f9f473391f14769951c95fa506fcefa => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:40.467007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6252cd-4a29-4293-9ff3-fcbdebf33f4c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:40.532006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:40.571553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6252cd-4a29-4293-9ff3-fcbdebf33f4c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:40.605829 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d1c36-2ed5-4470-8faf-2ebf3832b618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:40.606564 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d1c36-2ed5-4470-8faf-2ebf3832b618 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:40.607328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d1c36-2ed5-4470-8faf-2ebf3832b618 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:40.607611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d1c36-2ed5-4470-8faf-2ebf3832b618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:40.681436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d1c36-2ed5-4470-8faf-2ebf3832b618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:40.683294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d1c36-2ed5-4470-8faf-2ebf3832b618 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:40.747224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:40.788006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 156/316] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:40.793288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445f98c-9c2e-42f5-afc7-22af34b9735f None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:40.895256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445f98c-9c2e-42f5-afc7-22af34b9735f None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:40.929199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9eb8f-8acc-4623-8826-808281b488a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:40.930059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9eb8f-8acc-4623-8826-808281b488a8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:40.930343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9eb8f-8acc-4623-8826-808281b488a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:40.930618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9eb8f-8acc-4623-8826-808281b488a8 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:40.931808 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9eb8f-8acc-4623-8826-808281b488a8 None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:40.933274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9eb8f-8acc-4623-8826-808281b488a8 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:40.935845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 161/317] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:40 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:40.941775 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d1c36-2ed5-4470-8faf-2ebf3832b618 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:41.050307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d1c36-2ed5-4470-8faf-2ebf3832b618 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:41.085763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8f10e-1641-4783-839f-849591bd8cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:41.086470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8f10e-1641-4783-839f-849591bd8cae None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:41.086757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8f10e-1641-4783-839f-849591bd8cae None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:41.087035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8f10e-1641-4783-839f-849591bd8cae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:41.087864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8f10e-1641-4783-839f-849591bd8cae None admin] RBAC: Adding query filter params (role.id=b3b7dfbee8c34760b5acbb12576c8fdd, scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d, user.id=5f9f473391f14769951c95fa506fcefa) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:41.088205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8f10e-1641-4783-839f-849591bd8cae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:41.089688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8f10e-1641-4783-839f-849591bd8cae None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:41.096397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 157/318] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:06:40 2022] GET /identity/v3/role_assignments?user.id=5f9f473391f14769951c95fa506fcefa&scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d&role.id=b3b7dfbee8c34760b5acbb12576c8fdd => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:41.241062 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc4687-2a4b-4e7f-9a78-60d8c322459d None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:41.241445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc4687-2a4b-4e7f-9a78-60d8c322459d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:41.241803 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc4687-2a4b-4e7f-9a78-60d8c322459d None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:41.243611 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 162/319] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:41.249180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa8bdc-69e4-445f-a912-d5011230a3b7 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:41.249473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa8bdc-69e4-445f-a912-d5011230a3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:41.249760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa8bdc-69e4-445f-a912-d5011230a3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:41.267784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-abfa8bdc-69e4-445f-a912-d5011230a3b7 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:41.269038 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-abfa8bdc-69e4-445f-a912-d5011230a3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:41.294827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:41.349284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 158/320] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:41.371394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc4687-2a4b-4e7f-9a78-60d8c322459d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:41.411448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:41.484491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc4687-2a4b-4e7f-9a78-60d8c322459d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:41.518469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21cc13-731a-4d26-8066-3ca891da6ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:41.519255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21cc13-731a-4d26-8066-3ca891da6ebc None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:41.519544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21cc13-731a-4d26-8066-3ca891da6ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:41.519822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21cc13-731a-4d26-8066-3ca891da6ebc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:41.521014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21cc13-731a-4d26-8066-3ca891da6ebc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:41.522445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21cc13-731a-4d26-8066-3ca891da6ebc None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:41.524249 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 163/321] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:06:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:41.530044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa8bdc-69e4-445f-a912-d5011230a3b7 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:41.627997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa8bdc-69e4-445f-a912-d5011230a3b7 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:41.662253 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548d192-dbc1-47d4-8924-9566bd61b212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:41.662925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548d192-dbc1-47d4-8924-9566bd61b212 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:41.663211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548d192-dbc1-47d4-8924-9566bd61b212 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:41.663488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548d192-dbc1-47d4-8924-9566bd61b212 None admin] PATH_INFO: `/v3/groups` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:41.664341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548d192-dbc1-47d4-8924-9566bd61b212 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:41.665793 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548d192-dbc1-47d4-8924-9566bd61b212 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:41.680770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 159/322] 10.222.0.33 () {60 vars in 1134 bytes} [Fri Jun 17 07:06:41 2022] POST /identity/v3/groups => generated 226 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:06:42.070425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50464bf5-a8e9-4cb9-8216-5ca869663d32 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:42.070961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50464bf5-a8e9-4cb9-8216-5ca869663d32 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:42.071039 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50464bf5-a8e9-4cb9-8216-5ca869663d32 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:42.072583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 164/323] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:42.078075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2638e1d-3ee3-4cb7-bdbf-48990f5ceff0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:42.078610 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2638e1d-3ee3-4cb7-bdbf-48990f5ceff0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:42.079109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2638e1d-3ee3-4cb7-bdbf-48990f5ceff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:42.097093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-d2638e1d-3ee3-4cb7-bdbf-48990f5ceff0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:42.098596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d2638e1d-3ee3-4cb7-bdbf-48990f5ceff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:42.112801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:42.175174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 160/324] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:42.200267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50464bf5-a8e9-4cb9-8216-5ca869663d32 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:42.258869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fde79-eace-4980-bca4-ff949665c140 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:42.259213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fde79-eace-4980-bca4-ff949665c140 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:42.259478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fde79-eace-4980-bca4-ff949665c140 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:42.261135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 161/325] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:06:42.266765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60958b61-fe38-4cc1-8e64-82d12b29d1c5 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:42.267052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60958b61-fe38-4cc1-8e64-82d12b29d1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:42.267331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60958b61-fe38-4cc1-8e64-82d12b29d1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:42.285868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-60958b61-fe38-4cc1-8e64-82d12b29d1c5 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:42.286769 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-60958b61-fe38-4cc1-8e64-82d12b29d1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:42.319353 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:42.324501 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50464bf5-a8e9-4cb9-8216-5ca869663d32 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:42.378777 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0077aa1-d504-45b0-b338-4d1ffc4a3780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:42.379893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0077aa1-d504-45b0-b338-4d1ffc4a3780 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:42.380217 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0077aa1-d504-45b0-b338-4d1ffc4a3780 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:42.380371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 162/326] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:42.380488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0077aa1-d504-45b0-b338-4d1ffc4a3780 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:42.399426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0077aa1-d504-45b0-b338-4d1ffc4a3780 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-a0077aa1-d504-45b0-b338-4d1ffc4a3780 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:42.407604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:42.408526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:42.409312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 165/327] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:42.413976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60958b61-fe38-4cc1-8e64-82d12b29d1c5 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:42.414737 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0077aa1-d504-45b0-b338-4d1ffc4a3780 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:42.557555 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:42.581346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0077aa1-d504-45b0-b338-4d1ffc4a3780 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:42.589814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60958b61-fe38-4cc1-8e64-82d12b29d1c5 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:42.619226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb90e0-77ba-44b0-9109-45096c07568d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:42.619936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb90e0-77ba-44b0-9109-45096c07568d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:42.620312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb90e0-77ba-44b0-9109-45096c07568d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:42.620593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb90e0-77ba-44b0-9109-45096c07568d None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:42.621448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb90e0-77ba-44b0-9109-45096c07568d None admin] RBAC: Adding query filter params (name=member) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:42.621757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb90e0-77ba-44b0-9109-45096c07568d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:42.623804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb90e0-77ba-44b0-9109-45096c07568d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:42.633890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 166/328] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:42.645472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb90e0-77ba-44b0-9109-45096c07568d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:42.658772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d178e7b-1cf6-4878-bdd7-d0adfc1c1f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:42.659604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d178e7b-1cf6-4878-bdd7-d0adfc1c1f5a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:42.664178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d178e7b-1cf6-4878-bdd7-d0adfc1c1f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:42.664178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d178e7b-1cf6-4878-bdd7-d0adfc1c1f5a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:42.691792 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:42.693650 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d178e7b-1cf6-4878-bdd7-d0adfc1c1f5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-2d178e7b-1cf6-4878-bdd7-d0adfc1c1f5a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:42.701664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:42.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:42.703368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:42.703368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:42.703368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:42.703368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:42.703368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:42.703368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:42.703368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:42.703368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:42.703368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:42.703368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:42.703368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:42.703368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:42.704474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 163/329] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:42.710780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d178e7b-1cf6-4878-bdd7-d0adfc1c1f5a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:42.790924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb90e0-77ba-44b0-9109-45096c07568d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:42.832147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf6833-93ba-4318-ab05-e9d61eb12c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:42.834747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf6833-93ba-4318-ab05-e9d61eb12c60 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:42.834747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf6833-93ba-4318-ab05-e9d61eb12c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:42.834747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf6833-93ba-4318-ab05-e9d61eb12c60 None admin] PATH_INFO: `/v3/users/37730a8df26140599c95f5fc74ff0f08` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:42.836141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf6833-93ba-4318-ab05-e9d61eb12c60 None admin] RBAC: Authorizing `identity:get_user(user_id=37730a8df26140599c95f5fc74ff0f08)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:42.837731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf6833-93ba-4318-ab05-e9d61eb12c60 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:42.839677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 167/330] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:42 2022] GET /identity/v3/users/37730a8df26140599c95f5fc74ff0f08 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:42.849330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf6833-93ba-4318-ab05-e9d61eb12c60 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:42.866570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d178e7b-1cf6-4878-bdd7-d0adfc1c1f5a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:42.920776 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05265447-2e1a-40fb-b9b6-1d881df838b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:42.921466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05265447-2e1a-40fb-b9b6-1d881df838b3 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:42.921750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05265447-2e1a-40fb-b9b6-1d881df838b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:42.922025 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05265447-2e1a-40fb-b9b6-1d881df838b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:42.922779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05265447-2e1a-40fb-b9b6-1d881df838b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:42.923085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05265447-2e1a-40fb-b9b6-1d881df838b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:42.924505 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05265447-2e1a-40fb-b9b6-1d881df838b3 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:42.936899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:42.944175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 164/331] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:42.949553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05265447-2e1a-40fb-b9b6-1d881df838b3 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:42.955302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:42.967069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf6833-93ba-4318-ab05-e9d61eb12c60 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:43.004484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40652a3f-a4f7-42ea-abb3-1906c0e7624c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:43.005311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40652a3f-a4f7-42ea-abb3-1906c0e7624c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:43.005601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40652a3f-a4f7-42ea-abb3-1906c0e7624c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:43.006219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40652a3f-a4f7-42ea-abb3-1906c0e7624c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:43.007260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40652a3f-a4f7-42ea-abb3-1906c0e7624c None admin] RBAC: Adding query filter params (role.id=ce36ba2b01744465803d0533b2a47592, user.id=37730a8df26140599c95f5fc74ff0f08, scope.system=all) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:43.007582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40652a3f-a4f7-42ea-abb3-1906c0e7624c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:43.009093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40652a3f-a4f7-42ea-abb3-1906c0e7624c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:43.022753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 168/332] 10.222.0.33 () {56 vars in 1308 bytes} [Fri Jun 17 07:06:42 2022] GET /identity/v3/role_assignments?user.id=37730a8df26140599c95f5fc74ff0f08&scope.system=all&role.id=ce36ba2b01744465803d0533b2a47592 => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:43.158238 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05265447-2e1a-40fb-b9b6-1d881df838b3 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:43.206966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947ddb2-bbc0-4d97-a9a7-8ec256937ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:43.207867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947ddb2-bbc0-4d97-a9a7-8ec256937ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:43.208171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947ddb2-bbc0-4d97-a9a7-8ec256937ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:43.208448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947ddb2-bbc0-4d97-a9a7-8ec256937ca0 None admin] PATH_INFO: `/v3/users/5ba45f1acde54b03b9054267da6fd632` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:43.209699 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947ddb2-bbc0-4d97-a9a7-8ec256937ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=5ba45f1acde54b03b9054267da6fd632)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:43.211178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947ddb2-bbc0-4d97-a9a7-8ec256937ca0 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:43.213205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 165/333] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:42 2022] GET /identity/v3/users/5ba45f1acde54b03b9054267da6fd632 => generated 293 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:43.219536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40652a3f-a4f7-42ea-abb3-1906c0e7624c None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:43.359142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40652a3f-a4f7-42ea-abb3-1906c0e7624c None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:43.401133 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac2073-0b06-4f30-86a3-67bda8d8e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:43.402327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac2073-0b06-4f30-86a3-67bda8d8e3b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:43.402663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac2073-0b06-4f30-86a3-67bda8d8e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:43.402943 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac2073-0b06-4f30-86a3-67bda8d8e3b2 None admin] PATH_INFO: `/v3/system/users/5ba45f1acde54b03b9054267da6fd632/roles/b3b7dfbee8c34760b5acbb12576c8fdd` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:43.408882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac2073-0b06-4f30-86a3-67bda8d8e3b2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5ba45f1acde54b03b9054267da6fd632, role_id=b3b7dfbee8c34760b5acbb12576c8fdd)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:43.410227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac2073-0b06-4f30-86a3-67bda8d8e3b2 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:43.421693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 169/334] 10.222.0.33 () {58 vars in 1334 bytes} [Fri Jun 17 07:06:43 2022] PUT /identity/v3/system/users/5ba45f1acde54b03b9054267da6fd632/roles/b3b7dfbee8c34760b5acbb12576c8fdd => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:44.631231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1728cd-5437-4b6c-b9a0-0b21d1944dae None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:44.631737 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1728cd-5437-4b6c-b9a0-0b21d1944dae None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:44.631812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1728cd-5437-4b6c-b9a0-0b21d1944dae None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:44.633442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 166/335] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:44.639397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c54cd-e87d-44fe-81d7-1203a7729363 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:44.639788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c54cd-e87d-44fe-81d7-1203a7729363 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:44.639953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c54cd-e87d-44fe-81d7-1203a7729363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:44.660795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ee1c54cd-e87d-44fe-81d7-1203a7729363 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:44.661976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ee1c54cd-e87d-44fe-81d7-1203a7729363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:44.674871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:44.759675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 170/336] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:44.800777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1728cd-5437-4b6c-b9a0-0b21d1944dae None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:44.810286 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc843d-a69a-4c48-a534-7f655d097d57 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:44.810574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc843d-a69a-4c48-a534-7f655d097d57 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:44.810853 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc843d-a69a-4c48-a534-7f655d097d57 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:44.814339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 171/337] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:44.817957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2054-12a3-4710-996a-8ec2ede4fe4b None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:44.818452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2054-12a3-4710-996a-8ec2ede4fe4b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:44.818935 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2054-12a3-4710-996a-8ec2ede4fe4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:44.848409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-905e2054-12a3-4710-996a-8ec2ede4fe4b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:44.850012 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-905e2054-12a3-4710-996a-8ec2ede4fe4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:44.934149 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:44.980897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 172/338] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:44.982309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb2d5c-f4c2-4d28-a65f-41db806c98b3 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:44.982581 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb2d5c-f4c2-4d28-a65f-41db806c98b3 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:44.982864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb2d5c-f4c2-4d28-a65f-41db806c98b3 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:44.984288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 173/339] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:44.993693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08d214-7cc3-42a7-8234-ae130f040b59 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:44.993985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08d214-7cc3-42a7-8234-ae130f040b59 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:44.994266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08d214-7cc3-42a7-8234-ae130f040b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:45.000297 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1728cd-5437-4b6c-b9a0-0b21d1944dae None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:45.027048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-2d08d214-7cc3-42a7-8234-ae130f040b59 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:45.030689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-2d08d214-7cc3-42a7-8234-ae130f040b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:45.071959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f7cee-f1d6-438f-8313-c75b707b9613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:45.073212 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f7cee-f1d6-438f-8313-c75b707b9613 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:45.073692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f7cee-f1d6-438f-8313-c75b707b9613 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:45.074161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f7cee-f1d6-438f-8313-c75b707b9613 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:45.105106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f7cee-f1d6-438f-8313-c75b707b9613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-c99f7cee-f1d6-438f-8313-c75b707b9613 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:45.113470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:45.114635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:45.115585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:45.115585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:45.115585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:45.115585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:45.115585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:45.115585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:45.115585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:45.115585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:45.115585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:45.115585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:45.115585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:45.115585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:45.120325 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 167/340] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:06:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:45.121231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f7cee-f1d6-438f-8313-c75b707b9613 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:45.129349 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 174/341] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:45.130525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08d214-7cc3-42a7-8234-ae130f040b59 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:45.133132 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:45.243102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08d214-7cc3-42a7-8234-ae130f040b59 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:45.279819 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621958d-2418-41b8-86c8-b5d983cd7cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:45.280854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621958d-2418-41b8-86c8-b5d983cd7cec None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:45.281149 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621958d-2418-41b8-86c8-b5d983cd7cec None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:45.281406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621958d-2418-41b8-86c8-b5d983cd7cec None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:45.282171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621958d-2418-41b8-86c8-b5d983cd7cec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:45.282478 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621958d-2418-41b8-86c8-b5d983cd7cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:45.283867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621958d-2418-41b8-86c8-b5d983cd7cec None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:45.293316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 175/342] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:06:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:45.294147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621958d-2418-41b8-86c8-b5d983cd7cec None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:45.362499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:45.382221 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f7cee-f1d6-438f-8313-c75b707b9613 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:45.402870 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621958d-2418-41b8-86c8-b5d983cd7cec None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:45.420437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:45.432636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8f571-e5b6-4861-8c0a-690c039b1435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:45.433456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8f571-e5b6-4861-8c0a-690c039b1435 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:45.436091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8f571-e5b6-4861-8c0a-690c039b1435 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:45.436376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8f571-e5b6-4861-8c0a-690c039b1435 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:45.458645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d0456-2ff9-4812-84b3-6fced484de57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:45.459469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d0456-2ff9-4812-84b3-6fced484de57 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:45.459756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d0456-2ff9-4812-84b3-6fced484de57 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:45.460027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d0456-2ff9-4812-84b3-6fced484de57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:45.461585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb8f571-e5b6-4861-8c0a-690c039b1435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-eeb8f571-e5b6-4861-8c0a-690c039b1435 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:45.469614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:45.470562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:45.471345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 168/343] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:45.472011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8f571-e5b6-4861-8c0a-690c039b1435 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:45.483321 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d0456-2ff9-4812-84b3-6fced484de57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-2f0d0456-2ff9-4812-84b3-6fced484de57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:45.491008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:45.493226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:45.495175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:45.495175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:45.495175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:45.495175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:45.495175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:45.495175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:45.495175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:45.495175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:45.495175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:45.495175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:45.495175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:45.495175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:45.502872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 176/344] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:45.503605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d0456-2ff9-4812-84b3-6fced484de57 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:45.660017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d0456-2ff9-4812-84b3-6fced484de57 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:45.688438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:45.691302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8f571-e5b6-4861-8c0a-690c039b1435 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:45.703446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b48a3-cba0-4810-8b4f-6039b9e0827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:45.704149 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b48a3-cba0-4810-8b4f-6039b9e0827d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:45.704437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b48a3-cba0-4810-8b4f-6039b9e0827d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:45.704712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b48a3-cba0-4810-8b4f-6039b9e0827d None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:45.705466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b48a3-cba0-4810-8b4f-6039b9e0827d None admin] RBAC: Adding query filter params (name=member) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:45.705775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b48a3-cba0-4810-8b4f-6039b9e0827d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:45.707165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b48a3-cba0-4810-8b4f-6039b9e0827d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:45.726636 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 177/345] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:45.727599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b48a3-cba0-4810-8b4f-6039b9e0827d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:45.735223 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b260a76-3c01-4200-8a53-3c3ce462f3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:45.735902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b260a76-3c01-4200-8a53-3c3ce462f3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:45.736248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b260a76-3c01-4200-8a53-3c3ce462f3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:45.736533 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b260a76-3c01-4200-8a53-3c3ce462f3e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:45.848756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:45.871212 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b260a76-3c01-4200-8a53-3c3ce462f3e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:45.873283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b260a76-3c01-4200-8a53-3c3ce462f3e6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:45.929899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b48a3-cba0-4810-8b4f-6039b9e0827d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:45.967522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5d83d-cbc1-45e3-9565-5cc77ee02aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:45.968323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5d83d-cbc1-45e3-9565-5cc77ee02aaa None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:45.968608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5d83d-cbc1-45e3-9565-5cc77ee02aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:45.968887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5d83d-cbc1-45e3-9565-5cc77ee02aaa None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:45.969629 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5d83d-cbc1-45e3-9565-5cc77ee02aaa None admin] RBAC: Adding query filter params (name=member) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:45.969936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5d83d-cbc1-45e3-9565-5cc77ee02aaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:45.971305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5d83d-cbc1-45e3-9565-5cc77ee02aaa None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:45.981589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 178/346] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:45.983071 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5d83d-cbc1-45e3-9565-5cc77ee02aaa None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:46.002029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 169/347] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:46.003448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6820fae-a242-45a7-9ba6-d4bb6e9c4dc5 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:46.003748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6820fae-a242-45a7-9ba6-d4bb6e9c4dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:46.004026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6820fae-a242-45a7-9ba6-d4bb6e9c4dc5 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:46.005333 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 170/348] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:46.006046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6820fae-a242-45a7-9ba6-d4bb6e9c4dc5 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:46.039763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:46.096956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5d83d-cbc1-45e3-9565-5cc77ee02aaa None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:46.104409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:46.118183 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6820fae-a242-45a7-9ba6-d4bb6e9c4dc5 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:46.135171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147749d-ae0b-4d79-878a-e0e3131354e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:46.135826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147749d-ae0b-4d79-878a-e0e3131354e1 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:46.136133 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147749d-ae0b-4d79-878a-e0e3131354e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:46.136417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147749d-ae0b-4d79-878a-e0e3131354e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:46.156333 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16473841-54e2-41db-ac72-a73605ae64b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:46.157213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16473841-54e2-41db-ac72-a73605ae64b2 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:46.157506 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16473841-54e2-41db-ac72-a73605ae64b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:46.157839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16473841-54e2-41db-ac72-a73605ae64b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:46.212175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147749d-ae0b-4d79-878a-e0e3131354e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:46.213676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147749d-ae0b-4d79-878a-e0e3131354e1 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:46.230791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16473841-54e2-41db-ac72-a73605ae64b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:46.232329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16473841-54e2-41db-ac72-a73605ae64b2 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:46.266714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:46.322901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 179/349] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:46.323837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147749d-ae0b-4d79-878a-e0e3131354e1 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:46.324914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:46.339894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 171/350] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:46.341456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971cafe-5359-4ef7-83ca-9a7bdc02467c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:46.341751 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971cafe-5359-4ef7-83ca-9a7bdc02467c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:46.342065 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971cafe-5359-4ef7-83ca-9a7bdc02467c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:46.359347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-1971cafe-5359-4ef7-83ca-9a7bdc02467c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:46.360528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1971cafe-5359-4ef7-83ca-9a7bdc02467c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:46.433848 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147749d-ae0b-4d79-878a-e0e3131354e1 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:46.446247 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 172/351] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:46.447107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971cafe-5359-4ef7-83ca-9a7bdc02467c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:46.487460 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d3a11-fa1a-4f59-8b8c-2a1dedec6474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:46.488403 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d3a11-fa1a-4f59-8b8c-2a1dedec6474 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:46.488685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d3a11-fa1a-4f59-8b8c-2a1dedec6474 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:46.488956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d3a11-fa1a-4f59-8b8c-2a1dedec6474 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:46.490148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d3a11-fa1a-4f59-8b8c-2a1dedec6474 None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:46.491677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d3a11-fa1a-4f59-8b8c-2a1dedec6474 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:46.493518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 180/352] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:46 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:46.494385 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d3a11-fa1a-4f59-8b8c-2a1dedec6474 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:46.532576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:46.557144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971cafe-5359-4ef7-83ca-9a7bdc02467c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:46.576690 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:46.595206 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5873863-316d-4ff9-af61-4872dc554e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:46.596097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5873863-316d-4ff9-af61-4872dc554e5c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:46.596412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5873863-316d-4ff9-af61-4872dc554e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:46.596689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5873863-316d-4ff9-af61-4872dc554e5c None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:46.597964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5873863-316d-4ff9-af61-4872dc554e5c None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:46.599430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5873863-316d-4ff9-af61-4872dc554e5c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:46.601345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 173/353] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:46 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:46.601936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5873863-316d-4ff9-af61-4872dc554e5c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:46.616012 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d3a11-fa1a-4f59-8b8c-2a1dedec6474 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:46.657735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f70d36-c108-4824-917e-2a03cb190e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:46.658610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f70d36-c108-4824-917e-2a03cb190e83 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:46.658893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f70d36-c108-4824-917e-2a03cb190e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:46.659168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f70d36-c108-4824-917e-2a03cb190e83 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:46.660575 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f70d36-c108-4824-917e-2a03cb190e83 None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:46.662022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f70d36-c108-4824-917e-2a03cb190e83 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:46.664134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 181/354] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:46 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:46.664959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f70d36-c108-4824-917e-2a03cb190e83 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:46.716528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5873863-316d-4ff9-af61-4872dc554e5c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:46.756863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e2910-f77d-4d82-b3a1-a624540d9151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:46.757760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e2910-f77d-4d82-b3a1-a624540d9151 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:46.758052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e2910-f77d-4d82-b3a1-a624540d9151 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:46.758337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e2910-f77d-4d82-b3a1-a624540d9151 None admin] PATH_INFO: `/v3/groups/9d1ad469474b418aa7684da9fe9a4663` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:46.765080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e2910-f77d-4d82-b3a1-a624540d9151 None admin] RBAC: Authorizing `identity:get_group(group_id=9d1ad469474b418aa7684da9fe9a4663)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:46.765669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:46.767003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e2910-f77d-4d82-b3a1-a624540d9151 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:46.768293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 174/355] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:46 2022] GET /identity/v3/groups/9d1ad469474b418aa7684da9fe9a4663 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:46.769124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e2910-f77d-4d82-b3a1-a624540d9151 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:46.787200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f70d36-c108-4824-917e-2a03cb190e83 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:46.834629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:46.836615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23979a-706c-4f28-a48d-b8436dff139a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:46.838297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23979a-706c-4f28-a48d-b8436dff139a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:46.838592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23979a-706c-4f28-a48d-b8436dff139a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:46.838871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23979a-706c-4f28-a48d-b8436dff139a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:46.852139 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23979a-706c-4f28-a48d-b8436dff139a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-cb23979a-706c-4f28-a48d-b8436dff139a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:46.861582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:46.862555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 182/356] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:06:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:46.864383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23979a-706c-4f28-a48d-b8436dff139a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:46.894874 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e2910-f77d-4d82-b3a1-a624540d9151 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:46.936645 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7bc05-6907-47a6-91fd-f74e96ff9670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:46.937513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7bc05-6907-47a6-91fd-f74e96ff9670 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:46.937799 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7bc05-6907-47a6-91fd-f74e96ff9670 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:46.938085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7bc05-6907-47a6-91fd-f74e96ff9670 None admin] PATH_INFO: `/v3/groups/9d1ad469474b418aa7684da9fe9a4663` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:46.939258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7bc05-6907-47a6-91fd-f74e96ff9670 None admin] RBAC: Authorizing `identity:get_group(group_id=9d1ad469474b418aa7684da9fe9a4663)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:46.940763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7bc05-6907-47a6-91fd-f74e96ff9670 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:46.942555 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 175/357] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:46 2022] GET /identity/v3/groups/9d1ad469474b418aa7684da9fe9a4663 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:46.943584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63144cd7-7245-4f53-bd20-4f35aee6af5b None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:46.943878 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63144cd7-7245-4f53-bd20-4f35aee6af5b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:46.944188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63144cd7-7245-4f53-bd20-4f35aee6af5b None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:46.945549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 176/358] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:46.946585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63144cd7-7245-4f53-bd20-4f35aee6af5b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:46.985729 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23979a-706c-4f28-a48d-b8436dff139a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:47.027533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9d50f-ed6d-4ab9-9d1a-e6c1170ca87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:47.029140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9d50f-ed6d-4ab9-9d1a-e6c1170ca87f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:47.029431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9d50f-ed6d-4ab9-9d1a-e6c1170ca87f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:47.029712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9d50f-ed6d-4ab9-9d1a-e6c1170ca87f None admin] PATH_INFO: `/v3/groups/9d1ad469474b418aa7684da9fe9a4663` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:47.031390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9d50f-ed6d-4ab9-9d1a-e6c1170ca87f None admin] RBAC: Authorizing `identity:get_group(group_id=9d1ad469474b418aa7684da9fe9a4663)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:47.032439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9d50f-ed6d-4ab9-9d1a-e6c1170ca87f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:47.034699 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 183/359] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:46 2022] GET /identity/v3/groups/9d1ad469474b418aa7684da9fe9a4663 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:47.036156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab435759-f6e2-4e6b-bb01-ad235f996ba3 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:47.038908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab435759-f6e2-4e6b-bb01-ad235f996ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:47.038908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab435759-f6e2-4e6b-bb01-ad235f996ba3 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:47.038908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 184/360] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:47.039408 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab435759-f6e2-4e6b-bb01-ad235f996ba3 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:47.054738 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:47.063368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63144cd7-7245-4f53-bd20-4f35aee6af5b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:47.085105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:47.113147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4fa17-b103-4113-8ff5-4ca74b5e249e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:47.113371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4fa17-b103-4113-8ff5-4ca74b5e249e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:47.113653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4fa17-b103-4113-8ff5-4ca74b5e249e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:47.113936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4fa17-b103-4113-8ff5-4ca74b5e249e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:47.115106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4fa17-b103-4113-8ff5-4ca74b5e249e None admin] RBAC: Adding query filter params (group.id=9d1ad469474b418aa7684da9fe9a4663, role.id=22d7b181c7e041329c59c3eb6b492cd7, scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:47.115426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4fa17-b103-4113-8ff5-4ca74b5e249e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:47.117084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4fa17-b103-4113-8ff5-4ca74b5e249e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:47.123650 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 177/361] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jun 17 07:06:46 2022] GET /identity/v3/role_assignments?group.id=9d1ad469474b418aa7684da9fe9a4663&scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5&role.id=22d7b181c7e041329c59c3eb6b492cd7 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:47.124631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4fa17-b103-4113-8ff5-4ca74b5e249e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:47.185763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab435759-f6e2-4e6b-bb01-ad235f996ba3 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:47.227720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917691d-5ddf-453b-aced-c72c513d5665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:47.228446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917691d-5ddf-453b-aced-c72c513d5665 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:47.228738 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917691d-5ddf-453b-aced-c72c513d5665 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:47.229038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917691d-5ddf-453b-aced-c72c513d5665 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:47.229907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917691d-5ddf-453b-aced-c72c513d5665 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:47.230220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917691d-5ddf-453b-aced-c72c513d5665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:47.231633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917691d-5ddf-453b-aced-c72c513d5665 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:47.242987 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 185/362] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:06:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:47.244056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4e9ca-1e7e-4dff-80d9-13f1512c4fb1 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:47.248633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4e9ca-1e7e-4dff-80d9-13f1512c4fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:47.248921 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4e9ca-1e7e-4dff-80d9-13f1512c4fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:47.269137 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4fa17-b103-4113-8ff5-4ca74b5e249e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:47.272567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-bac4e9ca-1e7e-4dff-80d9-13f1512c4fb1 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:47.274254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-bac4e9ca-1e7e-4dff-80d9-13f1512c4fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:47.317510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace261e7-9a97-49a1-a9fe-b5fc0c5877ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:47.318179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace261e7-9a97-49a1-a9fe-b5fc0c5877ff None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:47.318466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace261e7-9a97-49a1-a9fe-b5fc0c5877ff None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:47.318745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace261e7-9a97-49a1-a9fe-b5fc0c5877ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:47.319581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace261e7-9a97-49a1-a9fe-b5fc0c5877ff None admin] RBAC: Adding query filter params (group.id=9d1ad469474b418aa7684da9fe9a4663, role.id=ce36ba2b01744465803d0533b2a47592, scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:47.319902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace261e7-9a97-49a1-a9fe-b5fc0c5877ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:47.321462 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace261e7-9a97-49a1-a9fe-b5fc0c5877ff None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:47.330332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 178/363] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jun 17 07:06:47 2022] GET /identity/v3/role_assignments?group.id=9d1ad469474b418aa7684da9fe9a4663&scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d&role.id=ce36ba2b01744465803d0533b2a47592 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:47.330332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace261e7-9a97-49a1-a9fe-b5fc0c5877ff None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:47.368499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 186/364] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:47.373480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3db5c-ccbd-4896-853d-fdcf0fc3bec5 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:47.373811 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3db5c-ccbd-4896-853d-fdcf0fc3bec5 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:47.374038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3db5c-ccbd-4896-853d-fdcf0fc3bec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:47.407315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:47.408769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:47.416445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-bed3db5c-ccbd-4896-853d-fdcf0fc3bec5 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:47.417780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-bed3db5c-ccbd-4896-853d-fdcf0fc3bec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:47.489101 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace261e7-9a97-49a1-a9fe-b5fc0c5877ff None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:47.520157 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 187/365] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:47.521102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3db5c-ccbd-4896-853d-fdcf0fc3bec5 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:47.534757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071dbee-b4bf-441c-8e18-fa7b82602b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:47.535428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071dbee-b4bf-441c-8e18-fa7b82602b4e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:47.535724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071dbee-b4bf-441c-8e18-fa7b82602b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:47.536008 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071dbee-b4bf-441c-8e18-fa7b82602b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:47.537954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071dbee-b4bf-441c-8e18-fa7b82602b4e None admin] RBAC: Adding query filter params (group.id=9d1ad469474b418aa7684da9fe9a4663, role.id=ce36ba2b01744465803d0533b2a47592, scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:47.537954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071dbee-b4bf-441c-8e18-fa7b82602b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:47.539412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071dbee-b4bf-441c-8e18-fa7b82602b4e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:47.548651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 179/366] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jun 17 07:06:47 2022] GET /identity/v3/role_assignments?group.id=9d1ad469474b418aa7684da9fe9a4663&scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5&role.id=ce36ba2b01744465803d0533b2a47592 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:47.549491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071dbee-b4bf-441c-8e18-fa7b82602b4e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:47.670245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3db5c-ccbd-4896-853d-fdcf0fc3bec5 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:47.705278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071dbee-b4bf-441c-8e18-fa7b82602b4e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:47.711760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e2c73-8908-4511-a29f-d76c88aa73cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:47.712492 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e2c73-8908-4511-a29f-d76c88aa73cc None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:47.712777 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e2c73-8908-4511-a29f-d76c88aa73cc None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:47.713057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e2c73-8908-4511-a29f-d76c88aa73cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:47.716144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:47.720241 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:47.753169 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f200af2-0e3e-46f5-8487-702f6933f0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:47.753910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f200af2-0e3e-46f5-8487-702f6933f0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:47.754193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f200af2-0e3e-46f5-8487-702f6933f0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:47.754464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f200af2-0e3e-46f5-8487-702f6933f0b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:47.769762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f200af2-0e3e-46f5-8487-702f6933f0b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-7f200af2-0e3e-46f5-8487-702f6933f0b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:47.777867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:47.778937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:47.779758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 180/367] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:06:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:47.781236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f200af2-0e3e-46f5-8487-702f6933f0b6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:47.816462 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e2c73-8908-4511-a29f-d76c88aa73cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:47.817656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e2c73-8908-4511-a29f-d76c88aa73cc None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:47.902772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f200af2-0e3e-46f5-8487-702f6933f0b6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:47.925182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 188/368] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:47.925647 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e2c73-8908-4511-a29f-d76c88aa73cc None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:47.943497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41398cb-81bd-4bce-9fe3-d4ee653e1f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:47.944675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41398cb-81bd-4bce-9fe3-d4ee653e1f18 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:47.945201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41398cb-81bd-4bce-9fe3-d4ee653e1f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:47.945689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41398cb-81bd-4bce-9fe3-d4ee653e1f18 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:47.957432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41398cb-81bd-4bce-9fe3-d4ee653e1f18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:47.964198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-b41398cb-81bd-4bce-9fe3-d4ee653e1f18 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:47.967410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:47.968366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:47.969158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:47.969158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:47.969158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:47.969158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:47.969158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:47.969158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:47.969158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:47.969158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:47.969158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:47.969158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:47.969158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 07:06:47.969158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:47.971869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 181/369] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:47.972987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41398cb-81bd-4bce-9fe3-d4ee653e1f18 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:48.006313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:48.044883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e2c73-8908-4511-a29f-d76c88aa73cc None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:48.083502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbcc38-7d99-4392-8c3b-0702540d0c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:48.084244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbcc38-7d99-4392-8c3b-0702540d0c03 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:48.084531 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbcc38-7d99-4392-8c3b-0702540d0c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:48.084807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbcc38-7d99-4392-8c3b-0702540d0c03 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:48.085559 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbcc38-7d99-4392-8c3b-0702540d0c03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:48.085866 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbcc38-7d99-4392-8c3b-0702540d0c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:48.087117 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41398cb-81bd-4bce-9fe3-d4ee653e1f18 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:48.087229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbcc38-7d99-4392-8c3b-0702540d0c03 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:48.096634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 189/370] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:48.097371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbcc38-7d99-4392-8c3b-0702540d0c03 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:48.132309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc966b-0000-467f-a594-ecd3ea81a3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:48.133578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc966b-0000-467f-a594-ecd3ea81a3ce None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:48.134069 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc966b-0000-467f-a594-ecd3ea81a3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:48.134560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc966b-0000-467f-a594-ecd3ea81a3ce None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:48.136015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc966b-0000-467f-a594-ecd3ea81a3ce None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:48.137718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc966b-0000-467f-a594-ecd3ea81a3ce None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:48.139838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 182/371] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:47 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:48.141112 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc966b-0000-467f-a594-ecd3ea81a3ce None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:48.220534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:48.223870 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbcc38-7d99-4392-8c3b-0702540d0c03 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:48.264175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bfb5d-efae-4825-b5d1-af8f6d3322bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:48.264175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bfb5d-efae-4825-b5d1-af8f6d3322bd None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:48.264175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bfb5d-efae-4825-b5d1-af8f6d3322bd None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:48.264175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bfb5d-efae-4825-b5d1-af8f6d3322bd None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:48.264554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bfb5d-efae-4825-b5d1-af8f6d3322bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:48.264554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bfb5d-efae-4825-b5d1-af8f6d3322bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:48.265011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bfb5d-efae-4825-b5d1-af8f6d3322bd None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:48.278816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 190/372] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:48.279710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bfb5d-efae-4825-b5d1-af8f6d3322bd None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:48.284421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:48.292308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc966b-0000-467f-a594-ecd3ea81a3ce None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:48.362446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90e47c-fa95-4f4f-a9d7-79bbad372993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:48.363127 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90e47c-fa95-4f4f-a9d7-79bbad372993 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:48.363409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90e47c-fa95-4f4f-a9d7-79bbad372993 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:48.363691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90e47c-fa95-4f4f-a9d7-79bbad372993 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:48.391061 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bfb5d-efae-4825-b5d1-af8f6d3322bd None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:48.459770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112b21f-f740-4d87-96e1-89c1b6fb50d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:48.461034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112b21f-f740-4d87-96e1-89c1b6fb50d7 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:48.461509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112b21f-f740-4d87-96e1-89c1b6fb50d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:48.461957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112b21f-f740-4d87-96e1-89c1b6fb50d7 None admin] PATH_INFO: `/v3/groups/9d1ad469474b418aa7684da9fe9a4663` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:48.468900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112b21f-f740-4d87-96e1-89c1b6fb50d7 None admin] RBAC: Authorizing `identity:get_group(group_id=9d1ad469474b418aa7684da9fe9a4663)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:48.470553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112b21f-f740-4d87-96e1-89c1b6fb50d7 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:48.472735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 191/373] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:48 2022] GET /identity/v3/groups/9d1ad469474b418aa7684da9fe9a4663 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:48.473712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112b21f-f740-4d87-96e1-89c1b6fb50d7 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:48.488820 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90e47c-fa95-4f4f-a9d7-79bbad372993 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:48.490349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90e47c-fa95-4f4f-a9d7-79bbad372993 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:48.518213 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:48.626048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112b21f-f740-4d87-96e1-89c1b6fb50d7 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:48.643918 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:48.650159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 183/374] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:48.651063 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90e47c-fa95-4f4f-a9d7-79bbad372993 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:48.701618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c77dcb-7c6f-494b-a5c0-512680662a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:48.702503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c77dcb-7c6f-494b-a5c0-512680662a6f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:48.702789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c77dcb-7c6f-494b-a5c0-512680662a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:48.703066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c77dcb-7c6f-494b-a5c0-512680662a6f None admin] PATH_INFO: `/v3/users/5ba45f1acde54b03b9054267da6fd632` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:48.705002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c77dcb-7c6f-494b-a5c0-512680662a6f None admin] RBAC: Authorizing `identity:get_user(user_id=5ba45f1acde54b03b9054267da6fd632)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:48.706453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c77dcb-7c6f-494b-a5c0-512680662a6f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:48.708112 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 192/375] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:06:48 2022] GET /identity/v3/users/5ba45f1acde54b03b9054267da6fd632 => generated 293 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:48.708881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c77dcb-7c6f-494b-a5c0-512680662a6f None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:48.791607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90e47c-fa95-4f4f-a9d7-79bbad372993 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:48.831839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f5dff-97d2-49ba-b5a3-8c797114dc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:48.832547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f5dff-97d2-49ba-b5a3-8c797114dc7a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:48.832837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f5dff-97d2-49ba-b5a3-8c797114dc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:48.833116 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f5dff-97d2-49ba-b5a3-8c797114dc7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:48.833953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f5dff-97d2-49ba-b5a3-8c797114dc7a None admin] RBAC: Adding query filter params (group.id=9d1ad469474b418aa7684da9fe9a4663, role.id=22d7b181c7e041329c59c3eb6b492cd7, scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:48.834277 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f5dff-97d2-49ba-b5a3-8c797114dc7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:48.835733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f5dff-97d2-49ba-b5a3-8c797114dc7a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:48.842248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 184/376] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jun 17 07:06:48 2022] GET /identity/v3/role_assignments?group.id=9d1ad469474b418aa7684da9fe9a4663&scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d&role.id=22d7b181c7e041329c59c3eb6b492cd7 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:48.843060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f5dff-97d2-49ba-b5a3-8c797114dc7a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:48.848235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c77dcb-7c6f-494b-a5c0-512680662a6f None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:48.852532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:48.891892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a10de0-024b-4e87-8f1e-8918e4974c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:48.893000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a10de0-024b-4e87-8f1e-8918e4974c08 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:48.893285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a10de0-024b-4e87-8f1e-8918e4974c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:48.893560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a10de0-024b-4e87-8f1e-8918e4974c08 None admin] PATH_INFO: `/v3/projects/7107699553c04a40be061877b81300e3` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:48.896424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a10de0-024b-4e87-8f1e-8918e4974c08 None admin] RBAC: Authorizing `identity:get_project(project_id=7107699553c04a40be061877b81300e3)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:48.897964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a10de0-024b-4e87-8f1e-8918e4974c08 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:48.900183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 193/377] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:48 2022] GET /identity/v3/projects/7107699553c04a40be061877b81300e3 => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:48.906375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a10de0-024b-4e87-8f1e-8918e4974c08 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:49.024336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a10de0-024b-4e87-8f1e-8918e4974c08 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:49.088891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:49.110037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca43fd4-2e91-4597-bba7-ec0d6df15a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:49.110918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca43fd4-2e91-4597-bba7-ec0d6df15a82 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:49.111206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca43fd4-2e91-4597-bba7-ec0d6df15a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:49.111482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca43fd4-2e91-4597-bba7-ec0d6df15a82 None admin] PATH_INFO: `/v3/groups/b8f60a093eae4e2b9e57fa8f2bc6e1f5` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:49.124213 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca43fd4-2e91-4597-bba7-ec0d6df15a82 None admin] RBAC: Authorizing `identity:get_group(group_id=b8f60a093eae4e2b9e57fa8f2bc6e1f5)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:49.125684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca43fd4-2e91-4597-bba7-ec0d6df15a82 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:49.128758 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 194/378] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:48 2022] GET /identity/v3/groups/b8f60a093eae4e2b9e57fa8f2bc6e1f5 => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:49.134608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca43fd4-2e91-4597-bba7-ec0d6df15a82 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:49.164519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f5dff-97d2-49ba-b5a3-8c797114dc7a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:49.190200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:49.240423 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229ec6b-d6f2-4478-ac19-32ce68bcb667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:49.241104 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229ec6b-d6f2-4478-ac19-32ce68bcb667 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:49.241394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229ec6b-d6f2-4478-ac19-32ce68bcb667 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:49.241677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229ec6b-d6f2-4478-ac19-32ce68bcb667 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:49.242519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229ec6b-d6f2-4478-ac19-32ce68bcb667 None admin] RBAC: Adding query filter params (role.id=b3b7dfbee8c34760b5acbb12576c8fdd, user.id=5ba45f1acde54b03b9054267da6fd632, scope.system=all) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:49.242839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229ec6b-d6f2-4478-ac19-32ce68bcb667 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:49.244313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229ec6b-d6f2-4478-ac19-32ce68bcb667 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:49.272976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 185/379] 10.222.0.33 () {56 vars in 1308 bytes} [Fri Jun 17 07:06:48 2022] GET /identity/v3/role_assignments?user.id=5ba45f1acde54b03b9054267da6fd632&scope.system=all&role.id=b3b7dfbee8c34760b5acbb12576c8fdd => generated 510 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:49.307419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca43fd4-2e91-4597-bba7-ec0d6df15a82 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:49.345422 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15c612-82a6-4cdf-b15b-4e7b6881554c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:49.346017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15c612-82a6-4cdf-b15b-4e7b6881554c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:49.346305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15c612-82a6-4cdf-b15b-4e7b6881554c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:49.346587 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15c612-82a6-4cdf-b15b-4e7b6881554c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:49.347443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15c612-82a6-4cdf-b15b-4e7b6881554c None admin] RBAC: Adding query filter params (group.id=b8f60a093eae4e2b9e57fa8f2bc6e1f5, role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=7107699553c04a40be061877b81300e3) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:49.347763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15c612-82a6-4cdf-b15b-4e7b6881554c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:49.349317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15c612-82a6-4cdf-b15b-4e7b6881554c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:49.356212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 195/380] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jun 17 07:06:49 2022] GET /identity/v3/role_assignments?group.id=b8f60a093eae4e2b9e57fa8f2bc6e1f5&scope.project.id=7107699553c04a40be061877b81300e3&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:49.833400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13626658-3cd0-4c55-b888-e44c1936c8df None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:49.834086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13626658-3cd0-4c55-b888-e44c1936c8df None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:49.834086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13626658-3cd0-4c55-b888-e44c1936c8df None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:49.835548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 186/381] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:49.841656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40669efc-3679-4df8-8b01-9086ddddd5b3 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:49.842560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40669efc-3679-4df8-8b01-9086ddddd5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:49.843036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40669efc-3679-4df8-8b01-9086ddddd5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:49.869205 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-40669efc-3679-4df8-8b01-9086ddddd5b3 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:49.870785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-40669efc-3679-4df8-8b01-9086ddddd5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:49.950459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 196/382] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:49.993007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13626658-3cd0-4c55-b888-e44c1936c8df None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:50.134575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13626658-3cd0-4c55-b888-e44c1936c8df None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:50.158179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:50.175749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76ae46-304e-43a1-8eb5-ddaedd289cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:50.176649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76ae46-304e-43a1-8eb5-ddaedd289cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:50.176943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76ae46-304e-43a1-8eb5-ddaedd289cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:50.177226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76ae46-304e-43a1-8eb5-ddaedd289cf1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:50.190995 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76ae46-304e-43a1-8eb5-ddaedd289cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-ed76ae46-304e-43a1-8eb5-ddaedd289cf1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:50.199337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:50.200252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:50.201062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 187/383] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:06:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:50.208555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40669efc-3679-4df8-8b01-9086ddddd5b3 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:50.242571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:50.346771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40669efc-3679-4df8-8b01-9086ddddd5b3 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:50.413686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948e855-2395-4cda-8e34-fc9c0a5672bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:50.414731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948e855-2395-4cda-8e34-fc9c0a5672bd None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:50.415213 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948e855-2395-4cda-8e34-fc9c0a5672bd None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:50.415686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948e855-2395-4cda-8e34-fc9c0a5672bd None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:50.416654 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948e855-2395-4cda-8e34-fc9c0a5672bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:50.417158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948e855-2395-4cda-8e34-fc9c0a5672bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:50.418770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948e855-2395-4cda-8e34-fc9c0a5672bd None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:50.431758 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 197/384] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:06:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:50.437640 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76ae46-304e-43a1-8eb5-ddaedd289cf1 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:50.588972 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76ae46-304e-43a1-8eb5-ddaedd289cf1 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:50.637772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43b2d6-827e-4b19-bf81-41670520f8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:50.639095 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43b2d6-827e-4b19-bf81-41670520f8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:50.639378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43b2d6-827e-4b19-bf81-41670520f8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:50.639653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43b2d6-827e-4b19-bf81-41670520f8b2 None admin] PATH_INFO: `/v3/groups/9d1ad469474b418aa7684da9fe9a4663` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:50.640872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43b2d6-827e-4b19-bf81-41670520f8b2 None admin] RBAC: Authorizing `identity:get_group(group_id=9d1ad469474b418aa7684da9fe9a4663)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:50.642309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43b2d6-827e-4b19-bf81-41670520f8b2 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:50.644272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 188/385] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:50 2022] GET /identity/v3/groups/9d1ad469474b418aa7684da9fe9a4663 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:50.649776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948e855-2395-4cda-8e34-fc9c0a5672bd None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:50.659446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182a5ab-6c4a-4b7a-bc75-5280878c7e88 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:50.659735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182a5ab-6c4a-4b7a-bc75-5280878c7e88 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:50.660020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182a5ab-6c4a-4b7a-bc75-5280878c7e88 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:50.661513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 189/386] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:06:50.666475 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db34e5-11bf-4cc5-897b-1a0ee65f7b94 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:50.666759 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db34e5-11bf-4cc5-897b-1a0ee65f7b94 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:50.667037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db34e5-11bf-4cc5-897b-1a0ee65f7b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:50.688913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c6db34e5-11bf-4cc5-897b-1a0ee65f7b94 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:50.690231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c6db34e5-11bf-4cc5-897b-1a0ee65f7b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:50.749203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:50.753550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:50.769101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948e855-2395-4cda-8e34-fc9c0a5672bd None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:50.796479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 190/387] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:06:50.821696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c1c95-0416-496e-b028-dc61504428cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:50.822931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c1c95-0416-496e-b028-dc61504428cc None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:50.823410 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c1c95-0416-496e-b028-dc61504428cc None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:50.823899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c1c95-0416-496e-b028-dc61504428cc None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:50.825307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c1c95-0416-496e-b028-dc61504428cc None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:50.827489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c1c95-0416-496e-b028-dc61504428cc None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:50.829742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 198/388] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:50 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:50.835195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db34e5-11bf-4cc5-897b-1a0ee65f7b94 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:50.860715 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c1c95-0416-496e-b028-dc61504428cc None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:51.037080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c1c95-0416-496e-b028-dc61504428cc None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:51.049882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db34e5-11bf-4cc5-897b-1a0ee65f7b94 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:51.076662 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbb1f6-e2fa-4d57-9882-766c9112bbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:51.077818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbb1f6-e2fa-4d57-9882-766c9112bbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:51.078296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbb1f6-e2fa-4d57-9882-766c9112bbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:51.078762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbb1f6-e2fa-4d57-9882-766c9112bbc8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:51.089814 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbb1f6-e2fa-4d57-9882-766c9112bbc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-42bbb1f6-e2fa-4d57-9882-766c9112bbc8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:51.098406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:51.099285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:51.100086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:51.100086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:51.100086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:51.100086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:51.100086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:51.100086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:51.100086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:51.100086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:51.100086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:51.100086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:51.100086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:51.100086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:51.102805 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 199/389] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:51.108757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbb1f6-e2fa-4d57-9882-766c9112bbc8 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:51.115549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abf687-f4cb-4e7b-a407-77ab7b436f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:51.116539 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abf687-f4cb-4e7b-a407-77ab7b436f2d None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:51.116836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abf687-f4cb-4e7b-a407-77ab7b436f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:51.117174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abf687-f4cb-4e7b-a407-77ab7b436f2d None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5/groups/9d1ad469474b418aa7684da9fe9a4663/roles/22d7b181c7e041329c59c3eb6b492cd7` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:51.123571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7abf687-f4cb-4e7b-a407-77ab7b436f2d None admin] RBAC: Authorizing `identity:create_grant(project_id=f1a4c2e64b4747a29a5b7958456ce1f5, group_id=9d1ad469474b418aa7684da9fe9a4663, role_id=22d7b181c7e041329c59c3eb6b492cd7)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:51.125332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7abf687-f4cb-4e7b-a407-77ab7b436f2d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:51.133367 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:51.145594 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 191/390] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jun 17 07:06:50 2022] PUT /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5/groups/9d1ad469474b418aa7684da9fe9a4663/roles/22d7b181c7e041329c59c3eb6b492cd7 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:51.223934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbb1f6-e2fa-4d57-9882-766c9112bbc8 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:51.261938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13347bd8-7ee9-4511-b1fc-d7d3a18f4ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:51.262630 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13347bd8-7ee9-4511-b1fc-d7d3a18f4ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:51.262916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13347bd8-7ee9-4511-b1fc-d7d3a18f4ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:51.263224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13347bd8-7ee9-4511-b1fc-d7d3a18f4ec0 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:51.264008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13347bd8-7ee9-4511-b1fc-d7d3a18f4ec0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:51.264355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13347bd8-7ee9-4511-b1fc-d7d3a18f4ec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:51.266082 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13347bd8-7ee9-4511-b1fc-d7d3a18f4ec0 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:51.275706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 200/391] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:51.281847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abf687-f4cb-4e7b-a407-77ab7b436f2d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:51.332559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:51.403607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abf687-f4cb-4e7b-a407-77ab7b436f2d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:51.440017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31745a-8dfb-427d-98a6-cf044dac9bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:51.440903 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31745a-8dfb-427d-98a6-cf044dac9bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:51.441188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31745a-8dfb-427d-98a6-cf044dac9bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:51.441463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31745a-8dfb-427d-98a6-cf044dac9bb0 None admin] PATH_INFO: `/v3/groups/9d1ad469474b418aa7684da9fe9a4663` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:51.442651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31745a-8dfb-427d-98a6-cf044dac9bb0 None admin] RBAC: Authorizing `identity:get_group(group_id=9d1ad469474b418aa7684da9fe9a4663)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:51.444128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31745a-8dfb-427d-98a6-cf044dac9bb0 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:51.446040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 192/392] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:51 2022] GET /identity/v3/groups/9d1ad469474b418aa7684da9fe9a4663 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:51.452016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f41b0-6b92-46d2-a6ee-a3fe76c19e1d None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:51.452330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f41b0-6b92-46d2-a6ee-a3fe76c19e1d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:51.452612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f41b0-6b92-46d2-a6ee-a3fe76c19e1d None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:51.454143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 201/393] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:51.454919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31745a-8dfb-427d-98a6-cf044dac9bb0 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:51.461348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa678904-6c7c-4767-95b8-a3d3bac4dd36 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:51.461646 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa678904-6c7c-4767-95b8-a3d3bac4dd36 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:51.461937 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa678904-6c7c-4767-95b8-a3d3bac4dd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:51.497126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-aa678904-6c7c-4767-95b8-a3d3bac4dd36 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:51.498898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-aa678904-6c7c-4767-95b8-a3d3bac4dd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:51.601645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 202/394] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:51.606547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31745a-8dfb-427d-98a6-cf044dac9bb0 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:51.624324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:51.627108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa678904-6c7c-4767-95b8-a3d3bac4dd36 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:51.629934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:51.683501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebb03e-1a6e-428a-a400-84aaa5599418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:51.684872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebb03e-1a6e-428a-a400-84aaa5599418 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:51.685386 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebb03e-1a6e-428a-a400-84aaa5599418 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:51.685863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebb03e-1a6e-428a-a400-84aaa5599418 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:51.687426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebb03e-1a6e-428a-a400-84aaa5599418 None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:51.689344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebb03e-1a6e-428a-a400-84aaa5599418 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:51.691362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 193/395] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:51 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:51.692951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042affd-18bd-4c2b-a8b3-084d471f83dd None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:51.693457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042affd-18bd-4c2b-a8b3-084d471f83dd None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:51.694011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042affd-18bd-4c2b-a8b3-084d471f83dd None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:51.695934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 194/396] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:51.697889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042affd-18bd-4c2b-a8b3-084d471f83dd None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:51.759169 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa678904-6c7c-4767-95b8-a3d3bac4dd36 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:51.794497 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801503ab-6260-4dd5-9328-623d9f542b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:51.795643 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801503ab-6260-4dd5-9328-623d9f542b6f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:51.795643 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801503ab-6260-4dd5-9328-623d9f542b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:51.795942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801503ab-6260-4dd5-9328-623d9f542b6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:51.806825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801503ab-6260-4dd5-9328-623d9f542b6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-801503ab-6260-4dd5-9328-623d9f542b6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:51.815518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:51.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 203/397] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:51.822335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f1f0b-8660-4a69-b69d-1b10808644c0 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:51.823083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f1f0b-8660-4a69-b69d-1b10808644c0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:51.823083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f1f0b-8660-4a69-b69d-1b10808644c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:51.825038 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042affd-18bd-4c2b-a8b3-084d471f83dd None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:51.836200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f73f1f0b-8660-4a69-b69d-1b10808644c0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:51.837446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f73f1f0b-8660-4a69-b69d-1b10808644c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:51.865828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bc225-c68d-47ab-8d66-c1f7bd2ea776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:51.867479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bc225-c68d-47ab-8d66-c1f7bd2ea776 None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:51.867979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bc225-c68d-47ab-8d66-c1f7bd2ea776 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:51.868722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bc225-c68d-47ab-8d66-c1f7bd2ea776 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d/groups/9d1ad469474b418aa7684da9fe9a4663/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:51.871099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bc225-c68d-47ab-8d66-c1f7bd2ea776 None admin] RBAC: Authorizing `identity:create_grant(project_id=36fac836e8de4d52aa468bc5deaf2a2d, group_id=9d1ad469474b418aa7684da9fe9a4663, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:51.873677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bc225-c68d-47ab-8d66-c1f7bd2ea776 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:51.889399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 195/398] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jun 17 07:06:51 2022] PUT /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d/groups/9d1ad469474b418aa7684da9fe9a4663/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:06:51.890782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bc225-c68d-47ab-8d66-c1f7bd2ea776 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:51.896897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:51.922480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 204/399] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:51.936719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:51.959527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f1f0b-8660-4a69-b69d-1b10808644c0 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:52.017360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bc225-c68d-47ab-8d66-c1f7bd2ea776 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:52.055884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e1600-eefd-46c4-a557-cb0fab1e279f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:52.057232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e1600-eefd-46c4-a557-cb0fab1e279f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:52.057934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e1600-eefd-46c4-a557-cb0fab1e279f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:52.058633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e1600-eefd-46c4-a557-cb0fab1e279f None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:52.060759 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e1600-eefd-46c4-a557-cb0fab1e279f None admin] RBAC: Adding query filter params (name=member) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:52.061314 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e1600-eefd-46c4-a557-cb0fab1e279f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:52.062932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e1600-eefd-46c4-a557-cb0fab1e279f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:52.080239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 196/400] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:52.081909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e1600-eefd-46c4-a557-cb0fab1e279f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:52.144130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f1f0b-8660-4a69-b69d-1b10808644c0 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:52.196573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:52.203665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e1600-eefd-46c4-a557-cb0fab1e279f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:52.208167 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a719ec-d0de-47d8-bbaf-28e9d9ca7d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:52.208991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a719ec-d0de-47d8-bbaf-28e9d9ca7d10 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:52.209278 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a719ec-d0de-47d8-bbaf-28e9d9ca7d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:52.209553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a719ec-d0de-47d8-bbaf-28e9d9ca7d10 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:52.221471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a719ec-d0de-47d8-bbaf-28e9d9ca7d10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-29a719ec-d0de-47d8-bbaf-28e9d9ca7d10 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:52.230938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:52.231830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:52.233704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 205/401] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:06:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:52.234617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75ec86-5163-4e24-8941-8c763b668b37 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:52.234617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75ec86-5163-4e24-8941-8c763b668b37 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:52.234925 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75ec86-5163-4e24-8941-8c763b668b37 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:52.236207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 206/402] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:52.239697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75ec86-5163-4e24-8941-8c763b668b37 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:52.249657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3343c1-0305-4b02-b0b3-88599f8f0892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:52.250525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3343c1-0305-4b02-b0b3-88599f8f0892 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:52.250805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3343c1-0305-4b02-b0b3-88599f8f0892 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:52.251077 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3343c1-0305-4b02-b0b3-88599f8f0892 None admin] PATH_INFO: `/v3/groups/9d1ad469474b418aa7684da9fe9a4663` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:52.252679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3343c1-0305-4b02-b0b3-88599f8f0892 None admin] RBAC: Authorizing `identity:get_group(group_id=9d1ad469474b418aa7684da9fe9a4663)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:52.253751 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3343c1-0305-4b02-b0b3-88599f8f0892 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:52.255382 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 197/403] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:52 2022] GET /identity/v3/groups/9d1ad469474b418aa7684da9fe9a4663 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:52.256558 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057416e4-bab4-4b16-be20-0e5673bfbaed None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:52.256845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057416e4-bab4-4b16-be20-0e5673bfbaed None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:52.257135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057416e4-bab4-4b16-be20-0e5673bfbaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:52.275165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-057416e4-bab4-4b16-be20-0e5673bfbaed None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:52.276356 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-057416e4-bab4-4b16-be20-0e5673bfbaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:52.317980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:52.357285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 198/404] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:52.358741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057416e4-bab4-4b16-be20-0e5673bfbaed None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:52.398546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75ec86-5163-4e24-8941-8c763b668b37 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:52.436724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabbd8e-6b90-4ff2-aa10-18abe80a9fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:52.437564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabbd8e-6b90-4ff2-aa10-18abe80a9fda None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:52.438275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabbd8e-6b90-4ff2-aa10-18abe80a9fda None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:52.438396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabbd8e-6b90-4ff2-aa10-18abe80a9fda None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:52.439221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabbd8e-6b90-4ff2-aa10-18abe80a9fda None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:52.439625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabbd8e-6b90-4ff2-aa10-18abe80a9fda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:52.441332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabbd8e-6b90-4ff2-aa10-18abe80a9fda None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:52.455141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 207/405] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:06:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:52.455141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabbd8e-6b90-4ff2-aa10-18abe80a9fda None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:52.488118 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:52.495021 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057416e4-bab4-4b16-be20-0e5673bfbaed None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:52.532161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475caaa-88fd-4444-96b8-1826071dee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:52.533032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475caaa-88fd-4444-96b8-1826071dee6b None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:52.533322 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475caaa-88fd-4444-96b8-1826071dee6b None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:52.533603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475caaa-88fd-4444-96b8-1826071dee6b None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:52.534989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475caaa-88fd-4444-96b8-1826071dee6b None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:52.536434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475caaa-88fd-4444-96b8-1826071dee6b None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:52.538327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 199/406] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:52 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:52.538922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475caaa-88fd-4444-96b8-1826071dee6b None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:52.573154 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabbd8e-6b90-4ff2-aa10-18abe80a9fda None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:52.588205 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:52.611548 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8416da1-bcd9-4571-933e-8ad4405934f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:52.612881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8416da1-bcd9-4571-933e-8ad4405934f0 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:52.613458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8416da1-bcd9-4571-933e-8ad4405934f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:52.613995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8416da1-bcd9-4571-933e-8ad4405934f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:52.625657 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8416da1-bcd9-4571-933e-8ad4405934f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-c8416da1-bcd9-4571-933e-8ad4405934f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:52.634477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:52.635433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:52.636620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:52.636620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:52.636620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:52.636620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:52.636620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:52.636620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:52.636620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:52.636620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:52.636620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:52.636620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:52.636620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:52.636620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:52.639659 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 208/407] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:06:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:52.640788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8416da1-bcd9-4571-933e-8ad4405934f0 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:52.656251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475caaa-88fd-4444-96b8-1826071dee6b None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:52.694006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a2914-b538-4d18-bca7-0d13eb2bf129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:52.694968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a2914-b538-4d18-bca7-0d13eb2bf129 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:52.695259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a2914-b538-4d18-bca7-0d13eb2bf129 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:52.695538 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a2914-b538-4d18-bca7-0d13eb2bf129 None admin] PATH_INFO: `/v3/groups/9d1ad469474b418aa7684da9fe9a4663` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:52.696997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a2914-b538-4d18-bca7-0d13eb2bf129 None admin] RBAC: Authorizing `identity:get_group(group_id=9d1ad469474b418aa7684da9fe9a4663)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:52.698441 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a2914-b538-4d18-bca7-0d13eb2bf129 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:52.705329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 200/408] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:52 2022] GET /identity/v3/groups/9d1ad469474b418aa7684da9fe9a4663 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:52.706277 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a2914-b538-4d18-bca7-0d13eb2bf129 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:52.739398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:52.752687 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8416da1-bcd9-4571-933e-8ad4405934f0 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:52.789187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b0b9e-7c5b-4677-ab47-42ade1d11375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:52.790417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b0b9e-7c5b-4677-ab47-42ade1d11375 None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:52.790898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b0b9e-7c5b-4677-ab47-42ade1d11375 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:52.791369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b0b9e-7c5b-4677-ab47-42ade1d11375 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5/groups/9d1ad469474b418aa7684da9fe9a4663/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:52.793710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b0b9e-7c5b-4677-ab47-42ade1d11375 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1a4c2e64b4747a29a5b7958456ce1f5, group_id=9d1ad469474b418aa7684da9fe9a4663, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:52.796045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b0b9e-7c5b-4677-ab47-42ade1d11375 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:52.812700 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 209/409] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jun 17 07:06:52 2022] PUT /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5/groups/9d1ad469474b418aa7684da9fe9a4663/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:52.814011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b0b9e-7c5b-4677-ab47-42ade1d11375 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:52.830051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a2914-b538-4d18-bca7-0d13eb2bf129 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:52.869270 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d7a05-de5a-4f1c-9f4c-923bc857b29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:52.870031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d7a05-de5a-4f1c-9f4c-923bc857b29c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:52.870297 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d7a05-de5a-4f1c-9f4c-923bc857b29c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:52.870624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d7a05-de5a-4f1c-9f4c-923bc857b29c None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:52.871387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d7a05-de5a-4f1c-9f4c-923bc857b29c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:52.871697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d7a05-de5a-4f1c-9f4c-923bc857b29c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:52.873648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d7a05-de5a-4f1c-9f4c-923bc857b29c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:52.882357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:52.885040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 201/410] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:52.891684 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d7a05-de5a-4f1c-9f4c-923bc857b29c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:52.992541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b0b9e-7c5b-4677-ab47-42ade1d11375 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:52.997210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:53.007184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d7a05-de5a-4f1c-9f4c-923bc857b29c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:53.033553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343ca36-05bf-4759-b9e3-cd424cec4d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:53.034930 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343ca36-05bf-4759-b9e3-cd424cec4d85 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:53.035410 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343ca36-05bf-4759-b9e3-cd424cec4d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:53.035909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343ca36-05bf-4759-b9e3-cd424cec4d85 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:53.037390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343ca36-05bf-4759-b9e3-cd424cec4d85 None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:53.039060 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343ca36-05bf-4759-b9e3-cd424cec4d85 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:53.041564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 210/411] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:52 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:53.049196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343ca36-05bf-4759-b9e3-cd424cec4d85 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:53.054779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8923498-6d84-4746-8344-886a0bd6ea0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:53.055665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8923498-6d84-4746-8344-886a0bd6ea0c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:53.055956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8923498-6d84-4746-8344-886a0bd6ea0c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:53.056266 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8923498-6d84-4746-8344-886a0bd6ea0c None admin] PATH_INFO: `/v3/groups/b8f60a093eae4e2b9e57fa8f2bc6e1f5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:53.057923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8923498-6d84-4746-8344-886a0bd6ea0c None admin] RBAC: Authorizing `identity:get_group(group_id=b8f60a093eae4e2b9e57fa8f2bc6e1f5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:53.059403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8923498-6d84-4746-8344-886a0bd6ea0c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:53.066677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 202/412] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:52 2022] GET /identity/v3/groups/b8f60a093eae4e2b9e57fa8f2bc6e1f5 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:53.073539 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8923498-6d84-4746-8344-886a0bd6ea0c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:53.204205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8923498-6d84-4746-8344-886a0bd6ea0c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:53.240440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594f3fa-b2c1-4cfc-85aa-edfc18749503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:53.240440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594f3fa-b2c1-4cfc-85aa-edfc18749503 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:53.240440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594f3fa-b2c1-4cfc-85aa-edfc18749503 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:53.240440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594f3fa-b2c1-4cfc-85aa-edfc18749503 None admin] PATH_INFO: `/v3/projects/7107699553c04a40be061877b81300e3` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:53.240854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594f3fa-b2c1-4cfc-85aa-edfc18749503 None admin] RBAC: Authorizing `identity:get_project(project_id=7107699553c04a40be061877b81300e3)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:53.240854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594f3fa-b2c1-4cfc-85aa-edfc18749503 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:53.241792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 203/413] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:53 2022] GET /identity/v3/projects/7107699553c04a40be061877b81300e3 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:53.247420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594f3fa-b2c1-4cfc-85aa-edfc18749503 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:53.267774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343ca36-05bf-4759-b9e3-cd424cec4d85 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:53.270778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:53.300419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:53.342143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775c233-c576-4920-8231-11cae7d3682d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:53.346114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775c233-c576-4920-8231-11cae7d3682d None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:53.346397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775c233-c576-4920-8231-11cae7d3682d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:53.346675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775c233-c576-4920-8231-11cae7d3682d None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d/groups/9d1ad469474b418aa7684da9fe9a4663/roles/22d7b181c7e041329c59c3eb6b492cd7` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:53.352666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775c233-c576-4920-8231-11cae7d3682d None admin] RBAC: Authorizing `identity:create_grant(project_id=36fac836e8de4d52aa468bc5deaf2a2d, group_id=9d1ad469474b418aa7684da9fe9a4663, role_id=22d7b181c7e041329c59c3eb6b492cd7)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:53.354909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775c233-c576-4920-8231-11cae7d3682d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:53.379725 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 211/414] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jun 17 07:06:53 2022] PUT /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d/groups/9d1ad469474b418aa7684da9fe9a4663/roles/22d7b181c7e041329c59c3eb6b492cd7 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:53.421200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594f3fa-b2c1-4cfc-85aa-edfc18749503 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:53.463731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80900ca8-46b9-4e56-8047-17fef7ba7b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:53.464973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80900ca8-46b9-4e56-8047-17fef7ba7b0e None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:53.465414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80900ca8-46b9-4e56-8047-17fef7ba7b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:53.465842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80900ca8-46b9-4e56-8047-17fef7ba7b0e None admin] PATH_INFO: `/v3/projects/7107699553c04a40be061877b81300e3/groups/b8f60a093eae4e2b9e57fa8f2bc6e1f5/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:53.468315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80900ca8-46b9-4e56-8047-17fef7ba7b0e None admin] RBAC: Authorizing `identity:create_grant(project_id=7107699553c04a40be061877b81300e3, group_id=b8f60a093eae4e2b9e57fa8f2bc6e1f5, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:53.470036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80900ca8-46b9-4e56-8047-17fef7ba7b0e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:53.492100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 204/415] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jun 17 07:06:53 2022] PUT /identity/v3/projects/7107699553c04a40be061877b81300e3/groups/b8f60a093eae4e2b9e57fa8f2bc6e1f5/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:06:53.558011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba990c-dcd4-4a19-b3bf-cb0c66a35d89 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:53.558190 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba990c-dcd4-4a19-b3bf-cb0c66a35d89 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:53.558484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba990c-dcd4-4a19-b3bf-cb0c66a35d89 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:53.560325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 212/416] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:53.565974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f263f-1cee-479a-bdc0-11f608b34ffb None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:53.566608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f263f-1cee-479a-bdc0-11f608b34ffb None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:53.567103 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f263f-1cee-479a-bdc0-11f608b34ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:53.591354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-7e6f263f-1cee-479a-bdc0-11f608b34ffb None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:53.593076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7e6f263f-1cee-479a-bdc0-11f608b34ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:53.687016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 205/417] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:53.709377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba990c-dcd4-4a19-b3bf-cb0c66a35d89 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:53.853454 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba990c-dcd4-4a19-b3bf-cb0c66a35d89 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:53.902427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a74307-7efc-4e5e-a913-c8d862f39ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:53.903255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a74307-7efc-4e5e-a913-c8d862f39ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:53.903544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a74307-7efc-4e5e-a913-c8d862f39ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:53.904382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a74307-7efc-4e5e-a913-c8d862f39ce6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:53.924967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a74307-7efc-4e5e-a913-c8d862f39ce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-36a74307-7efc-4e5e-a913-c8d862f39ce6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:53.933096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:53.933979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:53.934761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 213/418] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:06:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:53.940953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f263f-1cee-479a-bdc0-11f608b34ffb None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:53.965020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:54.079363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f263f-1cee-479a-bdc0-11f608b34ffb None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:54.115667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4dd82-1988-48ec-b3b7-933bd0e357e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:54.116722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4dd82-1988-48ec-b3b7-933bd0e357e0 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:54.117227 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4dd82-1988-48ec-b3b7-933bd0e357e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:54.117704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4dd82-1988-48ec-b3b7-933bd0e357e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:54.118655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4dd82-1988-48ec-b3b7-933bd0e357e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:54.119165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4dd82-1988-48ec-b3b7-933bd0e357e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:54.120818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4dd82-1988-48ec-b3b7-933bd0e357e0 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:54.129721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 206/419] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:06:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:54.135265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a74307-7efc-4e5e-a913-c8d862f39ce6 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:54.173404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:54.261769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a74307-7efc-4e5e-a913-c8d862f39ce6 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:54.307520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665eada3-f43d-4cd9-a6c1-c3f046593345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:54.308212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665eada3-f43d-4cd9-a6c1-c3f046593345 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:54.308502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665eada3-f43d-4cd9-a6c1-c3f046593345 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:54.308785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665eada3-f43d-4cd9-a6c1-c3f046593345 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:54.387332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665eada3-f43d-4cd9-a6c1-c3f046593345 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:54.387332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665eada3-f43d-4cd9-a6c1-c3f046593345 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:54.429543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:54.530142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 214/420] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:54.535323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4dd82-1988-48ec-b3b7-933bd0e357e0 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:54.668622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:54.696251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4dd82-1988-48ec-b3b7-933bd0e357e0 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:54.744776 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc9952-63ba-4662-9008-a6474caec80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:54.745669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc9952-63ba-4662-9008-a6474caec80d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:54.745959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc9952-63ba-4662-9008-a6474caec80d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:54.746240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc9952-63ba-4662-9008-a6474caec80d None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:54.748138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc9952-63ba-4662-9008-a6474caec80d None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:54.749632 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc9952-63ba-4662-9008-a6474caec80d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:54.751781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 207/421] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:54 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:54.757467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665eada3-f43d-4cd9-a6c1-c3f046593345 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:54.776031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a63247-3184-4213-84c2-613f6579a6c9 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:54.776234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a63247-3184-4213-84c2-613f6579a6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:54.776514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a63247-3184-4213-84c2-613f6579a6c9 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:54.778142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 208/422] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:54.790446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646593e-e9ee-452f-89d1-f03c91e16b84 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:54.790746 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646593e-e9ee-452f-89d1-f03c91e16b84 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:54.791035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646593e-e9ee-452f-89d1-f03c91e16b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:54.824637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-1646593e-e9ee-452f-89d1-f03c91e16b84 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:54.827805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1646593e-e9ee-452f-89d1-f03c91e16b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:54.908430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665eada3-f43d-4cd9-a6c1-c3f046593345 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:54.947845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:54.953747 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 209/423] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:54.964327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c6dbe-2569-4279-af92-7e6e54742d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:54.965546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c6dbe-2569-4279-af92-7e6e54742d55 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:54.966063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c6dbe-2569-4279-af92-7e6e54742d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:54.967116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c6dbe-2569-4279-af92-7e6e54742d55 None admin] PATH_INFO: `/v3/groups/9d1ad469474b418aa7684da9fe9a4663` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:54.968521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c6dbe-2569-4279-af92-7e6e54742d55 None admin] RBAC: Authorizing `identity:get_group(group_id=9d1ad469474b418aa7684da9fe9a4663)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:54.970251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c6dbe-2569-4279-af92-7e6e54742d55 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:54.972446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 215/424] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:54 2022] GET /identity/v3/groups/9d1ad469474b418aa7684da9fe9a4663 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:54.978678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646593e-e9ee-452f-89d1-f03c91e16b84 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:54.979685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c6dbe-2569-4279-af92-7e6e54742d55 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:55.090276 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:55.128864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646593e-e9ee-452f-89d1-f03c91e16b84 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:55.166862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c6dbe-2569-4279-af92-7e6e54742d55 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:55.181308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d02b2-7657-4b8a-9a21-ca51b6f1eae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:55.182240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d02b2-7657-4b8a-9a21-ca51b6f1eae5 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:55.182731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d02b2-7657-4b8a-9a21-ca51b6f1eae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:55.183210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d02b2-7657-4b8a-9a21-ca51b6f1eae5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:55.184291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d02b2-7657-4b8a-9a21-ca51b6f1eae5 None admin] RBAC: Adding query filter params (group.id=9d1ad469474b418aa7684da9fe9a4663, role.id=22d7b181c7e041329c59c3eb6b492cd7, scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:55.184863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d02b2-7657-4b8a-9a21-ca51b6f1eae5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:55.186667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d02b2-7657-4b8a-9a21-ca51b6f1eae5 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:55.193329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 210/425] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jun 17 07:06:54 2022] GET /identity/v3/role_assignments?group.id=9d1ad469474b418aa7684da9fe9a4663&scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5&role.id=22d7b181c7e041329c59c3eb6b492cd7 => generated 611 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:55.229451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb87aa5-957c-45cb-98aa-7b16550eab88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:55.230193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb87aa5-957c-45cb-98aa-7b16550eab88 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:55.230481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb87aa5-957c-45cb-98aa-7b16550eab88 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:55.230762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb87aa5-957c-45cb-98aa-7b16550eab88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:55.254310 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb87aa5-957c-45cb-98aa-7b16550eab88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-0fb87aa5-957c-45cb-98aa-7b16550eab88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:55.262273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:55.263262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:55.264078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:55.264078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:55.264078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:55.264078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:55.264078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:55.264078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:55.264078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:55.264078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:55.264078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:55.264078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:55.264078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:55.264078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:55.268579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 216/426] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:55.274880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d02b2-7657-4b8a-9a21-ca51b6f1eae5 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:55.426493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d02b2-7657-4b8a-9a21-ca51b6f1eae5 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:55.463013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5e33b-7262-407a-a10e-933c03480951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:55.463697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5e33b-7262-407a-a10e-933c03480951 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:55.463991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5e33b-7262-407a-a10e-933c03480951 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:55.464313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5e33b-7262-407a-a10e-933c03480951 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:55.465070 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5e33b-7262-407a-a10e-933c03480951 None admin] RBAC: Adding query filter params (name=member) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:55.465387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5e33b-7262-407a-a10e-933c03480951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:55.467285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5e33b-7262-407a-a10e-933c03480951 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:55.476413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 211/427] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:55.482613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb87aa5-957c-45cb-98aa-7b16550eab88 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:55.542972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:55.610396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb87aa5-957c-45cb-98aa-7b16550eab88 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:55.633417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e80695-b3ad-469f-bf3b-efa8d55965f0 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:55.633719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e80695-b3ad-469f-bf3b-efa8d55965f0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:55.633980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e80695-b3ad-469f-bf3b-efa8d55965f0 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:55.635589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 212/428] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:06:55.641273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81401436-dee5-49b7-9d34-e9e745ba76ed None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:55.641570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81401436-dee5-49b7-9d34-e9e745ba76ed None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:55.641854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81401436-dee5-49b7-9d34-e9e745ba76ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:55.647381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978841a-5efe-46d5-836b-0b67eaf0c5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:55.649660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978841a-5efe-46d5-836b-0b67eaf0c5fe None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:55.649660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978841a-5efe-46d5-836b-0b67eaf0c5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:55.649660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978841a-5efe-46d5-836b-0b67eaf0c5fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:55.651040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:55.671501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-81401436-dee5-49b7-9d34-e9e745ba76ed None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:55.673020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-81401436-dee5-49b7-9d34-e9e745ba76ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:55.725846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978841a-5efe-46d5-836b-0b67eaf0c5fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:55.727373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978841a-5efe-46d5-836b-0b67eaf0c5fe None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:55.760801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 213/429] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:55.783669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81401436-dee5-49b7-9d34-e9e745ba76ed None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:55.786369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:55.844825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 217/430] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:55.847468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978841a-5efe-46d5-836b-0b67eaf0c5fe None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:55.915054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81401436-dee5-49b7-9d34-e9e745ba76ed None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:55.963620 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc9a99-33b3-47fd-bf75-8da3b5d8501c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:55.964492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc9a99-33b3-47fd-bf75-8da3b5d8501c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:55.964774 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc9a99-33b3-47fd-bf75-8da3b5d8501c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:55.965051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc9a99-33b3-47fd-bf75-8da3b5d8501c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:55.973964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978841a-5efe-46d5-836b-0b67eaf0c5fe None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:55.979830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc9a99-33b3-47fd-bf75-8da3b5d8501c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-5cdc9a99-33b3-47fd-bf75-8da3b5d8501c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:55.989980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:55.990899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:55.991732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:55.991732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:55.991732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:55.991732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:55.991732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:55.991732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:55.991732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:55.991732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:55.991732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:55.991732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:55.991732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 07:06:55.991732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:55.992922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:55.996115 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 214/431] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:06:55.997324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc9a99-33b3-47fd-bf75-8da3b5d8501c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:56.024360 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69b2f0-b5b6-4a67-8a7b-e5f3a37a4936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:56.025138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69b2f0-b5b6-4a67-8a7b-e5f3a37a4936 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.025424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69b2f0-b5b6-4a67-8a7b-e5f3a37a4936 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.025702 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69b2f0-b5b6-4a67-8a7b-e5f3a37a4936 None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.027136 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69b2f0-b5b6-4a67-8a7b-e5f3a37a4936 None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:56.029201 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69b2f0-b5b6-4a67-8a7b-e5f3a37a4936 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:56.030890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 218/432] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:55 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:56.032044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed6e43-fa50-4794-8400-f80e398f31c0 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.032355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed6e43-fa50-4794-8400-f80e398f31c0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.032637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed6e43-fa50-4794-8400-f80e398f31c0 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.033850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 219/433] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:56.034916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857173e-1ec8-493d-af72-e607df63362f None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.035204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857173e-1ec8-493d-af72-e607df63362f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.035481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857173e-1ec8-493d-af72-e607df63362f None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.036718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 220/434] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:06:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:06:56.037415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857173e-1ec8-493d-af72-e607df63362f None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:56.055239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:56.103811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc9a99-33b3-47fd-bf75-8da3b5d8501c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:56.135321 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857173e-1ec8-493d-af72-e607df63362f None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:56.136363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6f0b4-5ffd-4c5a-9dac-8ebb81044306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:56.137034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6f0b4-5ffd-4c5a-9dac-8ebb81044306 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.137318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6f0b4-5ffd-4c5a-9dac-8ebb81044306 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.137603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6f0b4-5ffd-4c5a-9dac-8ebb81044306 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.138891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6f0b4-5ffd-4c5a-9dac-8ebb81044306 None admin] RBAC: Adding query filter params (name=member) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:56.139202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6f0b4-5ffd-4c5a-9dac-8ebb81044306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:56.140624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6f0b4-5ffd-4c5a-9dac-8ebb81044306 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:56.148540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 215/435] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:06:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:56.149846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2067d-591f-4cd0-b7e9-1d8a34641d62 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.150140 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2067d-591f-4cd0-b7e9-1d8a34641d62 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.150500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2067d-591f-4cd0-b7e9-1d8a34641d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.170051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-34a2067d-591f-4cd0-b7e9-1d8a34641d62 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:56.170228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68948e4-8a57-4255-90ad-cb04e3c79615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:56.171070 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68948e4-8a57-4255-90ad-cb04e3c79615 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.171350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68948e4-8a57-4255-90ad-cb04e3c79615 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.171624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68948e4-8a57-4255-90ad-cb04e3c79615 None admin] PATH_INFO: `/v3/groups/9d1ad469474b418aa7684da9fe9a4663` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.171936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-34a2067d-591f-4cd0-b7e9-1d8a34641d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:56.172796 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68948e4-8a57-4255-90ad-cb04e3c79615 None admin] RBAC: Authorizing `identity:get_group(group_id=9d1ad469474b418aa7684da9fe9a4663)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:56.174222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68948e4-8a57-4255-90ad-cb04e3c79615 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:56.175824 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 221/436] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:56 2022] GET /identity/v3/groups/9d1ad469474b418aa7684da9fe9a4663 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:56.176977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ca266-a559-477c-862d-95dc309e7d47 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.177263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ca266-a559-477c-862d-95dc309e7d47 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.177536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ca266-a559-477c-862d-95dc309e7d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.193353 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ab2ca266-a559-477c-862d-95dc309e7d47 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:06:56.194510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ab2ca266-a559-477c-862d-95dc309e7d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:06:56.222916 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:56.240084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 216/437] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:56.240896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2067d-591f-4cd0-b7e9-1d8a34641d62 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:56.263233 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 222/438] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:06:56.264218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ca266-a559-477c-862d-95dc309e7d47 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:56.281352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:56.340283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2067d-591f-4cd0-b7e9-1d8a34641d62 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:56.362561 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ca266-a559-477c-862d-95dc309e7d47 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:56.374028 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89719407-122a-40bd-832a-36df1dc381bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:56.374678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89719407-122a-40bd-832a-36df1dc381bc None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.374968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89719407-122a-40bd-832a-36df1dc381bc None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.375252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89719407-122a-40bd-832a-36df1dc381bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.396709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3887fdb-039e-439a-92fa-024330ca93ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:56.397360 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3887fdb-039e-439a-92fa-024330ca93ac None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.397642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3887fdb-039e-439a-92fa-024330ca93ac None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.397916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3887fdb-039e-439a-92fa-024330ca93ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.398743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3887fdb-039e-439a-92fa-024330ca93ac None admin] RBAC: Adding query filter params (group.id=9d1ad469474b418aa7684da9fe9a4663, role.id=ce36ba2b01744465803d0533b2a47592, scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:56.399055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3887fdb-039e-439a-92fa-024330ca93ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:56.400638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3887fdb-039e-439a-92fa-024330ca93ac None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:56.407318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 223/439] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jun 17 07:06:56 2022] GET /identity/v3/role_assignments?group.id=9d1ad469474b418aa7684da9fe9a4663&scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d&role.id=ce36ba2b01744465803d0533b2a47592 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:56.408295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3887fdb-039e-439a-92fa-024330ca93ac None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:56.438210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:56.453785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89719407-122a-40bd-832a-36df1dc381bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:56.455283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89719407-122a-40bd-832a-36df1dc381bc None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:56.504897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:56.527730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3887fdb-039e-439a-92fa-024330ca93ac None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:56.567161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fa264-542e-42f5-b75a-c24d91ce54b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:56.567886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fa264-542e-42f5-b75a-c24d91ce54b4 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.572583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fa264-542e-42f5-b75a-c24d91ce54b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.572699 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 217/440] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:56.572796 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fa264-542e-42f5-b75a-c24d91ce54b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.572870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89719407-122a-40bd-832a-36df1dc381bc None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:56.586771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fa264-542e-42f5-b75a-c24d91ce54b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-904fa264-542e-42f5-b75a-c24d91ce54b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:56.595489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:56.596467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 224/441] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:06:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:56.600797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fa264-542e-42f5-b75a-c24d91ce54b4 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:56.662880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:56.677261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89719407-122a-40bd-832a-36df1dc381bc None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:56.702985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fa264-542e-42f5-b75a-c24d91ce54b4 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:56.710494 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead1e6b-18e3-4a4e-ba46-fed4206935d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:56.711296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead1e6b-18e3-4a4e-ba46-fed4206935d1 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.711582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead1e6b-18e3-4a4e-ba46-fed4206935d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.711864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead1e6b-18e3-4a4e-ba46-fed4206935d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.722402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead1e6b-18e3-4a4e-ba46-fed4206935d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-fead1e6b-18e3-4a4e-ba46-fed4206935d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:06:56.730466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:06:56.731372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:06:56.732283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 218/442] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:06:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:06:56.732964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead1e6b-18e3-4a4e-ba46-fed4206935d1 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:56.739108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b1d39-aabf-4e47-9df9-1e2fd3229041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:56.739962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b1d39-aabf-4e47-9df9-1e2fd3229041 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.740279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b1d39-aabf-4e47-9df9-1e2fd3229041 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.740557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b1d39-aabf-4e47-9df9-1e2fd3229041 None admin] PATH_INFO: `/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.741749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b1d39-aabf-4e47-9df9-1e2fd3229041 None admin] RBAC: Authorizing `identity:get_project(project_id=f1a4c2e64b4747a29a5b7958456ce1f5)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:56.743194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b1d39-aabf-4e47-9df9-1e2fd3229041 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:56.744888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 225/443] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:56 2022] GET /identity/v3/projects/f1a4c2e64b4747a29a5b7958456ce1f5 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:56.745632 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b1d39-aabf-4e47-9df9-1e2fd3229041 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:56.759385 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:56.829432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead1e6b-18e3-4a4e-ba46-fed4206935d1 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:56.846026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b1d39-aabf-4e47-9df9-1e2fd3229041 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:56.861475 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e29a93-fb50-4ea1-87b7-2082616d5777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:56.862141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e29a93-fb50-4ea1-87b7-2082616d5777 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.862428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e29a93-fb50-4ea1-87b7-2082616d5777 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.862714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e29a93-fb50-4ea1-87b7-2082616d5777 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.863455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e29a93-fb50-4ea1-87b7-2082616d5777 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:56.863768 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e29a93-fb50-4ea1-87b7-2082616d5777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:56.865182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e29a93-fb50-4ea1-87b7-2082616d5777 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:56.873467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 219/444] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:06:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:56.874248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e29a93-fb50-4ea1-87b7-2082616d5777 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:56.879352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ceabc-afd1-4464-a09f-ebc6275f58dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:56.880012 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ceabc-afd1-4464-a09f-ebc6275f58dd None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:56.880319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ceabc-afd1-4464-a09f-ebc6275f58dd None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:56.880591 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ceabc-afd1-4464-a09f-ebc6275f58dd None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:56.881331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ceabc-afd1-4464-a09f-ebc6275f58dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:56.881636 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ceabc-afd1-4464-a09f-ebc6275f58dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:56.882982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ceabc-afd1-4464-a09f-ebc6275f58dd None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:56.884549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:56.891195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 226/445] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:06:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:56.891793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ceabc-afd1-4464-a09f-ebc6275f58dd None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:56.958268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:56.975413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e29a93-fb50-4ea1-87b7-2082616d5777 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:56.990664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ceabc-afd1-4464-a09f-ebc6275f58dd None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:57.008262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b939cba-94d0-490c-b03b-845d32043434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:57.009122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b939cba-94d0-490c-b03b-845d32043434 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:57.009412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b939cba-94d0-490c-b03b-845d32043434 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:57.009693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b939cba-94d0-490c-b03b-845d32043434 None admin] PATH_INFO: `/v3/groups/9d1ad469474b418aa7684da9fe9a4663` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:57.010875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b939cba-94d0-490c-b03b-845d32043434 None admin] RBAC: Authorizing `identity:get_group(group_id=9d1ad469474b418aa7684da9fe9a4663)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:57.012342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b939cba-94d0-490c-b03b-845d32043434 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:57.013977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 220/446] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:56 2022] GET /identity/v3/groups/9d1ad469474b418aa7684da9fe9a4663 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:57.014733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b939cba-94d0-490c-b03b-845d32043434 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:57.028080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fa2f7-e547-4ef4-ae95-1cca68571086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:57.029291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa2f7-e547-4ef4-ae95-1cca68571086 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:57.029575 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa2f7-e547-4ef4-ae95-1cca68571086 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:57.029851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa2f7-e547-4ef4-ae95-1cca68571086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:57.094278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:57.100234 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fa2f7-e547-4ef4-ae95-1cca68571086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:57.101680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fa2f7-e547-4ef4-ae95-1cca68571086 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:57.118623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b939cba-94d0-490c-b03b-845d32043434 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:57.151207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce516e48-d8ef-4565-8250-26fc65960672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:57.152172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce516e48-d8ef-4565-8250-26fc65960672 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:57.152532 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce516e48-d8ef-4565-8250-26fc65960672 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:57.152826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce516e48-d8ef-4565-8250-26fc65960672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:57.166778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:57.200168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 227/447] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:57.201479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fa2f7-e547-4ef4-ae95-1cca68571086 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:57.219923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce516e48-d8ef-4565-8250-26fc65960672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:57.221418 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce516e48-d8ef-4565-8250-26fc65960672 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:57.295235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:57.298898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fa2f7-e547-4ef4-ae95-1cca68571086 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:57.319055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 221/448] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:06:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:06:57.319874 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce516e48-d8ef-4565-8250-26fc65960672 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:57.335900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef17ad-51f6-4dea-8d26-066d461ad7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:57.336588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef17ad-51f6-4dea-8d26-066d461ad7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:57.336881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef17ad-51f6-4dea-8d26-066d461ad7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:57.337164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef17ad-51f6-4dea-8d26-066d461ad7c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:57.338000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef17ad-51f6-4dea-8d26-066d461ad7c4 None admin] RBAC: Adding query filter params (group.id=9d1ad469474b418aa7684da9fe9a4663, role.id=ce36ba2b01744465803d0533b2a47592, scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:57.338316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef17ad-51f6-4dea-8d26-066d461ad7c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:57.339743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef17ad-51f6-4dea-8d26-066d461ad7c4 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:57.345635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 228/449] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jun 17 07:06:57 2022] GET /identity/v3/role_assignments?group.id=9d1ad469474b418aa7684da9fe9a4663&scope.project.id=f1a4c2e64b4747a29a5b7958456ce1f5&role.id=ce36ba2b01744465803d0533b2a47592 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:57.346345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef17ad-51f6-4dea-8d26-066d461ad7c4 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:57.388416 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:57.450398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce516e48-d8ef-4565-8250-26fc65960672 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:57.468122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef17ad-51f6-4dea-8d26-066d461ad7c4 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:57.486965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53061a97-3890-4370-ab77-669540fad9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:57.487841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53061a97-3890-4370-ab77-669540fad9fc None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:57.488154 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53061a97-3890-4370-ab77-669540fad9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:57.488437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53061a97-3890-4370-ab77-669540fad9fc None admin] PATH_INFO: `/v3/projects/7107699553c04a40be061877b81300e3` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:57.489688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53061a97-3890-4370-ab77-669540fad9fc None admin] RBAC: Authorizing `identity:get_project(project_id=7107699553c04a40be061877b81300e3)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:57.491176 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53061a97-3890-4370-ab77-669540fad9fc None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:57.493131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 222/450] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:57 2022] GET /identity/v3/projects/7107699553c04a40be061877b81300e3 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:57.499201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53061a97-3890-4370-ab77-669540fad9fc None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:57.506429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072e1bf-41c3-4ebf-81f7-79779a0f552d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:57.507313 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072e1bf-41c3-4ebf-81f7-79779a0f552d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:57.507597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072e1bf-41c3-4ebf-81f7-79779a0f552d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:57.507876 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072e1bf-41c3-4ebf-81f7-79779a0f552d None admin] PATH_INFO: `/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:57.509131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072e1bf-41c3-4ebf-81f7-79779a0f552d None admin] RBAC: Authorizing `identity:get_project(project_id=36fac836e8de4d52aa468bc5deaf2a2d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:57.511070 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072e1bf-41c3-4ebf-81f7-79779a0f552d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:57.513568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 229/451] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:06:57 2022] GET /identity/v3/projects/36fac836e8de4d52aa468bc5deaf2a2d => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:57.517871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072e1bf-41c3-4ebf-81f7-79779a0f552d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:57.555893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:57.619771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53061a97-3890-4370-ab77-669540fad9fc None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:57.636229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072e1bf-41c3-4ebf-81f7-79779a0f552d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:57.637981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:57.664121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b593a39-d76e-44a9-b979-b91d54980e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:57.664996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b593a39-d76e-44a9-b979-b91d54980e2d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:57.665288 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b593a39-d76e-44a9-b979-b91d54980e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:57.665575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b593a39-d76e-44a9-b979-b91d54980e2d None admin] PATH_INFO: `/v3/groups/b8f60a093eae4e2b9e57fa8f2bc6e1f5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:57.667627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b593a39-d76e-44a9-b979-b91d54980e2d None admin] RBAC: Authorizing `identity:get_group(group_id=b8f60a093eae4e2b9e57fa8f2bc6e1f5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:57.671455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b593a39-d76e-44a9-b979-b91d54980e2d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:57.673440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 223/452] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:57 2022] GET /identity/v3/groups/b8f60a093eae4e2b9e57fa8f2bc6e1f5 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:57.677337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc939f2a-9d25-460d-bef3-a56b4147f8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:57.678200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc939f2a-9d25-460d-bef3-a56b4147f8ea None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:57.679265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc939f2a-9d25-460d-bef3-a56b4147f8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:57.679265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc939f2a-9d25-460d-bef3-a56b4147f8ea None admin] PATH_INFO: `/v3/groups/9d1ad469474b418aa7684da9fe9a4663` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:57.679451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b593a39-d76e-44a9-b979-b91d54980e2d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:57.680285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc939f2a-9d25-460d-bef3-a56b4147f8ea None admin] RBAC: Authorizing `identity:get_group(group_id=9d1ad469474b418aa7684da9fe9a4663)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:57.681725 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc939f2a-9d25-460d-bef3-a56b4147f8ea None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:57.683423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 230/453] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jun 17 07:06:57 2022] GET /identity/v3/groups/9d1ad469474b418aa7684da9fe9a4663 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:06:57.689445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc939f2a-9d25-460d-bef3-a56b4147f8ea None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:06:57.797415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b593a39-d76e-44a9-b979-b91d54980e2d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:57.812249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:06:57.818048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc939f2a-9d25-460d-bef3-a56b4147f8ea None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:06:57.838540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb3d98-9eb8-483b-a4b3-814b22064d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:57.839211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb3d98-9eb8-483b-a4b3-814b22064d88 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:57.839498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb3d98-9eb8-483b-a4b3-814b22064d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:57.839777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb3d98-9eb8-483b-a4b3-814b22064d88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:57.840666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb3d98-9eb8-483b-a4b3-814b22064d88 None admin] RBAC: Adding query filter params (group.id=b8f60a093eae4e2b9e57fa8f2bc6e1f5, role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=7107699553c04a40be061877b81300e3) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:57.841504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb3d98-9eb8-483b-a4b3-814b22064d88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:57.842948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb3d98-9eb8-483b-a4b3-814b22064d88 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:57.849398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 224/454] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jun 17 07:06:57 2022] GET /identity/v3/role_assignments?group.id=b8f60a093eae4e2b9e57fa8f2bc6e1f5&scope.project.id=7107699553c04a40be061877b81300e3&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:06:57.857496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d15187-fa8d-4092-8a34-8d84d85cfff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:06:57.858174 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d15187-fa8d-4092-8a34-8d84d85cfff5 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:06:57.858460 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d15187-fa8d-4092-8a34-8d84d85cfff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:06:57.858737 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d15187-fa8d-4092-8a34-8d84d85cfff5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:06:57.859624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d15187-fa8d-4092-8a34-8d84d85cfff5 None admin] RBAC: Adding query filter params (group.id=9d1ad469474b418aa7684da9fe9a4663, role.id=22d7b181c7e041329c59c3eb6b492cd7, scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:06:57.859931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d15187-fa8d-4092-8a34-8d84d85cfff5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:06:57.861471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d15187-fa8d-4092-8a34-8d84d85cfff5 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:06:57.868481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 231/455] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jun 17 07:06:57 2022] GET /identity/v3/role_assignments?group.id=9d1ad469474b418aa7684da9fe9a4663&scope.project.id=36fac836e8de4d52aa468bc5deaf2a2d&role.id=22d7b181c7e041329c59c3eb6b492cd7 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:00.617500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafa6a3-3135-4590-a267-622e06291dba None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:00.618359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafa6a3-3135-4590-a267-622e06291dba None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:00.618778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafa6a3-3135-4590-a267-622e06291dba None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:00.620596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 225/456] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:00.626200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17913158-1208-4750-a6c0-7f4c8fd71f57 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:00.626932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17913158-1208-4750-a6c0-7f4c8fd71f57 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:00.627357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17913158-1208-4750-a6c0-7f4c8fd71f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:00.649187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-17913158-1208-4750-a6c0-7f4c8fd71f57 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:00.650327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-17913158-1208-4750-a6c0-7f4c8fd71f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:00.678508 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:00.741980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 232/457] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:00.779189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafa6a3-3135-4590-a267-622e06291dba None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:00.811761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363090e-0232-4a0f-ab77-8f91ce933756 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:00.812085 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363090e-0232-4a0f-ab77-8f91ce933756 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:00.812363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363090e-0232-4a0f-ab77-8f91ce933756 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:00.813976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 233/458] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:07:00.819050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ce570-9483-421a-b793-39d2ca76996a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:00.819569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ce570-9483-421a-b793-39d2ca76996a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:00.820050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ce570-9483-421a-b793-39d2ca76996a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:00.838975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-e74ce570-9483-421a-b793-39d2ca76996a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:00.841838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e74ce570-9483-421a-b793-39d2ca76996a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:00.929340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 234/459] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:07:00.951385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafa6a3-3135-4590-a267-622e06291dba None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:00.958570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ce570-9483-421a-b793-39d2ca76996a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:01.017639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0a50c-e661-44c9-8a42-879ae7d996cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:01.018931 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0a50c-e661-44c9-8a42-879ae7d996cf None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.019426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0a50c-e661-44c9-8a42-879ae7d996cf None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.019908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0a50c-e661-44c9-8a42-879ae7d996cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.035760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0a50c-e661-44c9-8a42-879ae7d996cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:01.037573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0a50c-e661-44c9-8a42-879ae7d996cf None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-64a0a50c-e661-44c9-8a42-879ae7d996cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:01.068222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:01.069146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:01.070075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:01.070075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:01.070075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:01.070075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:01.072381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 226/460] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:01.081401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0a50c-e661-44c9-8a42-879ae7d996cf None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:01.145318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:01.151866 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:01.161279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ce570-9483-421a-b793-39d2ca76996a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:01.210619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743bdaa-25c7-4837-8ef9-00b8ad29cd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:01.211782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743bdaa-25c7-4837-8ef9-00b8ad29cd9a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.212335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743bdaa-25c7-4837-8ef9-00b8ad29cd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.213343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743bdaa-25c7-4837-8ef9-00b8ad29cd9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.227960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743bdaa-25c7-4837-8ef9-00b8ad29cd9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:01.230237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743bdaa-25c7-4837-8ef9-00b8ad29cd9a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:01.243954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0a50c-e661-44c9-8a42-879ae7d996cf None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-2743bdaa-25c7-4837-8ef9-00b8ad29cd9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:01.252100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:01.252933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:01.253783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:01.253783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:01.253783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:01.253783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:01.253783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 235/461] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:01.254568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9359a-06c1-449a-bbdd-def6a4ddbc2f None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.254849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9359a-06c1-449a-bbdd-def6a4ddbc2f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.255224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9359a-06c1-449a-bbdd-def6a4ddbc2f None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.256618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 236/462] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:01.257651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6634a-98eb-48ce-b159-0d0e9d8a74dd None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.257945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6634a-98eb-48ce-b159-0d0e9d8a74dd None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.258215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6634a-98eb-48ce-b159-0d0e9d8a74dd None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.260229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 237/463] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:01.261491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6634a-98eb-48ce-b159-0d0e9d8a74dd None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:01.287876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bccd9dd-1ec6-4dca-b01b-c720fff2a0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:01.288674 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccd9dd-1ec6-4dca-b01b-c720fff2a0aa None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.288988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccd9dd-1ec6-4dca-b01b-c720fff2a0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.289243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccd9dd-1ec6-4dca-b01b-c720fff2a0aa None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.290210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccd9dd-1ec6-4dca-b01b-c720fff2a0aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:01.290324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccd9dd-1ec6-4dca-b01b-c720fff2a0aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:01.291646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccd9dd-1ec6-4dca-b01b-c720fff2a0aa None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:01.310331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 227/464] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:01.310331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dd260-9760-4719-8fca-64910257934f None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.310331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dd260-9760-4719-8fca-64910257934f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.310331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dd260-9760-4719-8fca-64910257934f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.326315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-d74dd260-9760-4719-8fca-64910257934f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:01.327497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d74dd260-9760-4719-8fca-64910257934f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:01.377138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6634a-98eb-48ce-b159-0d0e9d8a74dd None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:01.403980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 228/465] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:01.405665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd03380-2b8b-40e7-b9b0-37a26ce9d32b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.405957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd03380-2b8b-40e7-b9b0-37a26ce9d32b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.406237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd03380-2b8b-40e7-b9b0-37a26ce9d32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.420261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6a693-0b96-40d2-a947-1fd0fe5db807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:01.420932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6a693-0b96-40d2-a947-1fd0fe5db807 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.421219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6a693-0b96-40d2-a947-1fd0fe5db807 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.421498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6a693-0b96-40d2-a947-1fd0fe5db807 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.422222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6a693-0b96-40d2-a947-1fd0fe5db807 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:01.422907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6a693-0b96-40d2-a947-1fd0fe5db807 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:01.423893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6a693-0b96-40d2-a947-1fd0fe5db807 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:01.429891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-2cd03380-2b8b-40e7-b9b0-37a26ce9d32b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:01.431573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-2cd03380-2b8b-40e7-b9b0-37a26ce9d32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:01.441312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 238/466] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:01.442182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6a693-0b96-40d2-a947-1fd0fe5db807 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:01.446344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:01.453032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:01.512162 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 229/467] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:01.512162 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd03380-2b8b-40e7-b9b0-37a26ce9d32b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:01.556709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6a693-0b96-40d2-a947-1fd0fe5db807 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:01.592736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d134476-b1b2-4ecb-802c-2293fa9f67b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:01.593406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d134476-b1b2-4ecb-802c-2293fa9f67b9 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.593678 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d134476-b1b2-4ecb-802c-2293fa9f67b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.593957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d134476-b1b2-4ecb-802c-2293fa9f67b9 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.594656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d134476-b1b2-4ecb-802c-2293fa9f67b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:01.596091 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d134476-b1b2-4ecb-802c-2293fa9f67b9 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:01.609314 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd03380-2b8b-40e7-b9b0-37a26ce9d32b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:01.620097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 239/468] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:07:01 2022] POST /identity/v3/users => generated 246 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:01.620097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d134476-b1b2-4ecb-802c-2293fa9f67b9 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:01.660848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d71a4-77fc-4a5e-a3e1-5fc4de83a281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:01.661761 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d71a4-77fc-4a5e-a3e1-5fc4de83a281 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.662113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d71a4-77fc-4a5e-a3e1-5fc4de83a281 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.662467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d71a4-77fc-4a5e-a3e1-5fc4de83a281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.675105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d71a4-77fc-4a5e-a3e1-5fc4de83a281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:01.676665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d71a4-77fc-4a5e-a3e1-5fc4de83a281 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:01.686966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-999d71a4-77fc-4a5e-a3e1-5fc4de83a281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:01.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:01.700481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:01.701365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:01.701365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:01.701365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:01.701365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:01.701586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:01.701994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 230/469] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:01.708957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d71a4-77fc-4a5e-a3e1-5fc4de83a281 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:01.754833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d134476-b1b2-4ecb-802c-2293fa9f67b9 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:01.792282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a007310-a4ac-487b-acca-a1cf8bff74d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:01.792887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a007310-a4ac-487b-acca-a1cf8bff74d5 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.793171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a007310-a4ac-487b-acca-a1cf8bff74d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.793498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a007310-a4ac-487b-acca-a1cf8bff74d5 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.794715 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a007310-a4ac-487b-acca-a1cf8bff74d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:01.796211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a007310-a4ac-487b-acca-a1cf8bff74d5 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:01.817987 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 240/470] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:07:01 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:01.818917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a007310-a4ac-487b-acca-a1cf8bff74d5 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:01.835404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d71a4-77fc-4a5e-a3e1-5fc4de83a281 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:01.878111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc895f-0e45-4ec6-8cdc-3d808084216b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:01.879064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc895f-0e45-4ec6-8cdc-3d808084216b None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.879358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc895f-0e45-4ec6-8cdc-3d808084216b None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.879637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc895f-0e45-4ec6-8cdc-3d808084216b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.893201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc895f-0e45-4ec6-8cdc-3d808084216b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:01.894800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc895f-0e45-4ec6-8cdc-3d808084216b None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-97cc895f-0e45-4ec6-8cdc-3d808084216b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:01.919060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:01.919973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:01.920896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:01.920896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:01.920896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:01.920896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:01.920896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 231/471] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:01.927172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc895f-0e45-4ec6-8cdc-3d808084216b None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:01.934246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a007310-a4ac-487b-acca-a1cf8bff74d5 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:01.955064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:01.975539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a73a1a-8faf-46db-8bee-7cd30d895750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:01.976255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a73a1a-8faf-46db-8bee-7cd30d895750 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:01.976541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a73a1a-8faf-46db-8bee-7cd30d895750 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:01.976816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a73a1a-8faf-46db-8bee-7cd30d895750 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:01.977568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a73a1a-8faf-46db-8bee-7cd30d895750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:01.977874 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a73a1a-8faf-46db-8bee-7cd30d895750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:01.979275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a73a1a-8faf-46db-8bee-7cd30d895750 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:01.992729 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 241/472] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:01.998067 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a73a1a-8faf-46db-8bee-7cd30d895750 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:01.998240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:02.055965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc895f-0e45-4ec6-8cdc-3d808084216b None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:02.100006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a9d25-d6bb-4024-bbf5-83d6eda5060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:02.100763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a9d25-d6bb-4024-bbf5-83d6eda5060a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:02.101056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a9d25-d6bb-4024-bbf5-83d6eda5060a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:02.101338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a9d25-d6bb-4024-bbf5-83d6eda5060a None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:02.102072 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a9d25-d6bb-4024-bbf5-83d6eda5060a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:02.102387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a9d25-d6bb-4024-bbf5-83d6eda5060a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:02.103781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a9d25-d6bb-4024-bbf5-83d6eda5060a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:02.117024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 232/473] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:02.118360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad8a98-78a6-4788-a8d0-98845e2e911b None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:02.123493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad8a98-78a6-4788-a8d0-98845e2e911b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:02.123813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad8a98-78a6-4788-a8d0-98845e2e911b None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:02.129164 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 233/474] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:02.129164 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad8a98-78a6-4788-a8d0-98845e2e911b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:02.129897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a73a1a-8faf-46db-8bee-7cd30d895750 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:02.179625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b58eb9-edf1-46ba-9170-339cacc85db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:02.180563 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b58eb9-edf1-46ba-9170-339cacc85db0 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:02.180952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b58eb9-edf1-46ba-9170-339cacc85db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:02.181332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b58eb9-edf1-46ba-9170-339cacc85db0 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:02.182187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b58eb9-edf1-46ba-9170-339cacc85db0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:02.184959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b58eb9-edf1-46ba-9170-339cacc85db0 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:02.211695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 242/475] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:07:01 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:07:02.213192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd45e5c-251c-40a4-a93f-62d055a6ee3f None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:02.213610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd45e5c-251c-40a4-a93f-62d055a6ee3f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:02.214040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd45e5c-251c-40a4-a93f-62d055a6ee3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:02.239624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-cfd45e5c-251c-40a4-a93f-62d055a6ee3f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:02.241528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-cfd45e5c-251c-40a4-a93f-62d055a6ee3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:02.260510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad8a98-78a6-4788-a8d0-98845e2e911b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:02.279312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:02.295186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:02.338610 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d3b90-12f6-4467-9ccd-f50a6d4ba7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:02.339227 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d3b90-12f6-4467-9ccd-f50a6d4ba7ba None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:02.339510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d3b90-12f6-4467-9ccd-f50a6d4ba7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:02.339787 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d3b90-12f6-4467-9ccd-f50a6d4ba7ba None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:02.340583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d3b90-12f6-4467-9ccd-f50a6d4ba7ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:02.342459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d3b90-12f6-4467-9ccd-f50a6d4ba7ba None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:02.367862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 243/476] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:02.383280 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 234/477] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:07:02 2022] POST /identity/v3/users => generated 249 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:02.404325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd45e5c-251c-40a4-a93f-62d055a6ee3f None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:02.529152 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd45e5c-251c-40a4-a93f-62d055a6ee3f None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:02.567692 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739d169-67ea-494c-aa00-26e9040ea450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:02.568412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739d169-67ea-494c-aa00-26e9040ea450 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:02.568702 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739d169-67ea-494c-aa00-26e9040ea450 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:02.568978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739d169-67ea-494c-aa00-26e9040ea450 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:02.569738 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739d169-67ea-494c-aa00-26e9040ea450 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:02.571225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739d169-67ea-494c-aa00-26e9040ea450 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:02.583609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.sql.core [None req-2739d169-67ea-494c-aa00-26e9040ea450 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 17 07:07:02.583609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 17 07:07:02.583609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [parameters: {'id': 'e1b6f4dd3ab0447eb2746f4433ea59af', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 17 07:07:02.583609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80117) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-2739d169-67ea-494c-aa00-26e9040ea450 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application conn.query(q) Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application result.read() Jun 17 07:07:02.600786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return ref.to_dict() Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 07:07:02.601670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application next(self.gen) Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application yield resource Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application next(self.gen) Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self.session.rollback() Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self.force_reraise() Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise self.value Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application session.commit() Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 17 07:07:02.602593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._prepare_impl() Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self.session.flush() Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._flush(objects) Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application compat.raise_( Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application flush_context.execute() Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rec.execute(self) Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 17 07:07:02.603457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 07:07:02.605486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application conn.query(q) Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application result.read() Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 07:07:02.608001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [parameters: {'id': 'e1b6f4dd3ab0447eb2746f4433ea59af', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:02.609712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 17 07:07:02.610782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:02.611782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 244/478] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jun 17 07:07:02 2022] POST /identity/v3/roles => generated 144 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:02.611858 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d3b90-12f6-4467-9ccd-f50a6d4ba7ba None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:02.726524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d3b90-12f6-4467-9ccd-f50a6d4ba7ba None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:02.761374 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a4917-e2f6-4f4c-ae16-02d29367d006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:02.762706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a4917-e2f6-4f4c-ae16-02d29367d006 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:02.763006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a4917-e2f6-4f4c-ae16-02d29367d006 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:02.763287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a4917-e2f6-4f4c-ae16-02d29367d006 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:02.773967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a4917-e2f6-4f4c-ae16-02d29367d006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-c68a4917-e2f6-4f4c-ae16-02d29367d006 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:02.782510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:02.783575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:02.784828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 235/479] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:07:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:02.790854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739d169-67ea-494c-aa00-26e9040ea450 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:02.823718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:02.920885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739d169-67ea-494c-aa00-26e9040ea450 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:02.963013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9f9ba-05fa-47b8-b115-80d67b73fc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:02.963745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9f9ba-05fa-47b8-b115-80d67b73fc8c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:02.964040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9f9ba-05fa-47b8-b115-80d67b73fc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:02.964358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9f9ba-05fa-47b8-b115-80d67b73fc8c None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:02.965161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9f9ba-05fa-47b8-b115-80d67b73fc8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:02.966002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9f9ba-05fa-47b8-b115-80d67b73fc8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:02.967377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9f9ba-05fa-47b8-b115-80d67b73fc8c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:02.975700 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 245/480] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:04.239359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5a4db-1048-4cc1-849e-c64fddfc0800 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:04.240245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5a4db-1048-4cc1-849e-c64fddfc0800 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:04.240668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5a4db-1048-4cc1-849e-c64fddfc0800 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:04.242427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 236/481] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:04.248303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2e007-b48d-4aa2-8d93-567d933228c5 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:04.248772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2e007-b48d-4aa2-8d93-567d933228c5 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:04.248864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2e007-b48d-4aa2-8d93-567d933228c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:04.270159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-44a2e007-b48d-4aa2-8d93-567d933228c5 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:04.271344 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-44a2e007-b48d-4aa2-8d93-567d933228c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:04.355035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 246/482] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:04.388576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5a4db-1048-4cc1-849e-c64fddfc0800 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:04.432200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:04.532850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5a4db-1048-4cc1-849e-c64fddfc0800 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:04.535051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fef6c7-ef14-4912-bfb2-f63d01e542cc None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:04.535352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fef6c7-ef14-4912-bfb2-f63d01e542cc None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:04.535636 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fef6c7-ef14-4912-bfb2-f63d01e542cc None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:04.537363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 247/483] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:04.543196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc9104-3b3d-476d-ad5b-b9df9bfbb6d1 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:04.543519 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc9104-3b3d-476d-ad5b-b9df9bfbb6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:04.543799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc9104-3b3d-476d-ad5b-b9df9bfbb6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:04.570805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3310228-4889-42ec-a4d6-b7d02652a7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:04.571944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3310228-4889-42ec-a4d6-b7d02652a7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:04.572487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3310228-4889-42ec-a4d6-b7d02652a7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:04.572960 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3310228-4889-42ec-a4d6-b7d02652a7d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:04.584437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3310228-4889-42ec-a4d6-b7d02652a7d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:04.589084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-77fc9104-3b3d-476d-ad5b-b9df9bfbb6d1 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:04.590318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-77fc9104-3b3d-476d-ad5b-b9df9bfbb6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-f3310228-4889-42ec-a4d6-b7d02652a7d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:04.593329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:04.594193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:04.594993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:04.594993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:04.594993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:04.594993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:04.594993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:04.594993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:04.594993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:04.594993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:04.594993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:04.594993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:04.594993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:04.594993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:04.597659 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 237/484] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:04.603257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3310228-4889-42ec-a4d6-b7d02652a7d0 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:04.623558 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:04.710037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:04.728040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3310228-4889-42ec-a4d6-b7d02652a7d0 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:04.765019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 248/485] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:04.774691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac40c07-9378-4af7-84ce-198a410d1261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:04.775585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac40c07-9378-4af7-84ce-198a410d1261 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:04.776018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac40c07-9378-4af7-84ce-198a410d1261 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:04.776728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac40c07-9378-4af7-84ce-198a410d1261 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:04.777622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac40c07-9378-4af7-84ce-198a410d1261 None admin] RBAC: Adding query filter params (name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:04.778138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac40c07-9378-4af7-84ce-198a410d1261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:04.779663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac40c07-9378-4af7-84ce-198a410d1261 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:04.789116 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 238/486] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:04.794743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc9104-3b3d-476d-ad5b-b9df9bfbb6d1 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:04.813753 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac40c07-9378-4af7-84ce-198a410d1261 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:04.938310 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc9104-3b3d-476d-ad5b-b9df9bfbb6d1 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:04.977218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c4678-c2bb-4e0b-ab0f-3409340e3eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:04.978257 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c4678-c2bb-4e0b-ab0f-3409340e3eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:04.978935 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c4678-c2bb-4e0b-ab0f-3409340e3eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:04.979358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c4678-c2bb-4e0b-ab0f-3409340e3eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:04.991402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c4678-c2bb-4e0b-ab0f-3409340e3eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:04.993903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c4678-c2bb-4e0b-ab0f-3409340e3eb8 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-559c4678-c2bb-4e0b-ab0f-3409340e3eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:05.014685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:05.015766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:05.016861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:05.016861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:05.016861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:05.016861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:05.018679 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 249/487] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:05.020083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc272f-1bc1-436c-866d-f0aa5e6e154b None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.020518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc272f-1bc1-436c-866d-f0aa5e6e154b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.020938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc272f-1bc1-436c-866d-f0aa5e6e154b None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.022553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 250/488] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:05.023749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc272f-1bc1-436c-866d-f0aa5e6e154b None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:05.026977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:05.034397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac40c07-9378-4af7-84ce-198a410d1261 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:05.069544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c5900-2972-4744-9ee9-ed423882ae7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:05.070331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c5900-2972-4744-9ee9-ed423882ae7f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.070622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c5900-2972-4744-9ee9-ed423882ae7f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.070954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c5900-2972-4744-9ee9-ed423882ae7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.082463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c5900-2972-4744-9ee9-ed423882ae7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-667c5900-2972-4744-9ee9-ed423882ae7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:05.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:05.091719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:05.092728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 239/489] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:05.093453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01a3a7-0a40-486f-8259-eb034b9a6a56 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.093660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01a3a7-0a40-486f-8259-eb034b9a6a56 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.093943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01a3a7-0a40-486f-8259-eb034b9a6a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.111726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-dc01a3a7-0a40-486f-8259-eb034b9a6a56 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:05.113126 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-dc01a3a7-0a40-486f-8259-eb034b9a6a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:05.116227 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:05.136449 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc272f-1bc1-436c-866d-f0aa5e6e154b None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:05.177603 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7cfa6-3e28-4723-9c8d-2bc90567c246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:05.178675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7cfa6-3e28-4723-9c8d-2bc90567c246 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.179151 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7cfa6-3e28-4723-9c8d-2bc90567c246 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.179576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7cfa6-3e28-4723-9c8d-2bc90567c246 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.180643 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7cfa6-3e28-4723-9c8d-2bc90567c246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:05.181108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7cfa6-3e28-4723-9c8d-2bc90567c246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:05.182835 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7cfa6-3e28-4723-9c8d-2bc90567c246 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:05.195730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 251/490] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:05.196844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7cfa6-3e28-4723-9c8d-2bc90567c246 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:05.199909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 240/491] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:05.200904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01a3a7-0a40-486f-8259-eb034b9a6a56 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:05.270197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:05.316181 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7cfa6-3e28-4723-9c8d-2bc90567c246 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:05.324454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01a3a7-0a40-486f-8259-eb034b9a6a56 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:05.353656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8db039-4fef-471c-827f-d79c6da1e491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:05.354341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8db039-4fef-471c-827f-d79c6da1e491 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.354801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8db039-4fef-471c-827f-d79c6da1e491 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.355021 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8db039-4fef-471c-827f-d79c6da1e491 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.356579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8db039-4fef-471c-827f-d79c6da1e491 None admin] RBAC: Adding query filter params (name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:05.356684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8db039-4fef-471c-827f-d79c6da1e491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:05.358343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8db039-4fef-471c-827f-d79c6da1e491 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:05.363287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ecf01-8772-4933-a3ea-38875a35c05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:05.364520 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ecf01-8772-4933-a3ea-38875a35c05b None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.364817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ecf01-8772-4933-a3ea-38875a35c05b None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.365089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ecf01-8772-4933-a3ea-38875a35c05b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.371755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 252/492] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:05.372631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ecf01-8772-4933-a3ea-38875a35c05b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:05.373515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb17aa-e080-4831-b0a3-2c299a741812 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.373966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb17aa-e080-4831-b0a3-2c299a741812 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.374087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ecf01-8772-4933-a3ea-38875a35c05b None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:05.374592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb17aa-e080-4831-b0a3-2c299a741812 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.376240 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 253/493] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:05.376783 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:05.377896 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb17aa-e080-4831-b0a3-2c299a741812 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-cd6ecf01-8772-4933-a3ea-38875a35c05b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:05.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:05.397906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:05.398797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:05.398797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:05.398797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:05.398797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:05.398797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:05.398797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:05.398797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:05.398797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:05.398797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:07:05.398797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:05.398797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 241/494] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jun 17 07:07:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:05.398797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ecf01-8772-4933-a3ea-38875a35c05b None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:05.495123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb17aa-e080-4831-b0a3-2c299a741812 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:05.504613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ecf01-8772-4933-a3ea-38875a35c05b None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:05.519484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:05.532372 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222c1f-39fc-47df-9043-3cf76ee4222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:05.533166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222c1f-39fc-47df-9043-3cf76ee4222c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.533448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222c1f-39fc-47df-9043-3cf76ee4222c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.533725 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222c1f-39fc-47df-9043-3cf76ee4222c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.544341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04222c1f-39fc-47df-9043-3cf76ee4222c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:05.544483 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf38fdd-0800-4c02-8b8c-eae24b8ca0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:05.545227 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf38fdd-0800-4c02-8b8c-eae24b8ca0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.545521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf38fdd-0800-4c02-8b8c-eae24b8ca0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.545832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf38fdd-0800-4c02-8b8c-eae24b8ca0c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-04222c1f-39fc-47df-9043-3cf76ee4222c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:05.552096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:05.552951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 254/495] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:05.558772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9beed4-eb57-4f99-a2f2-1b92d5d3ee32 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.559671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9beed4-eb57-4f99-a2f2-1b92d5d3ee32 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.559671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9beed4-eb57-4f99-a2f2-1b92d5d3ee32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.564287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf38fdd-0800-4c02-8b8c-eae24b8ca0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:05.565840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf38fdd-0800-4c02-8b8c-eae24b8ca0c2 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:05.572601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ce9beed4-eb57-4f99-a2f2-1b92d5d3ee32 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:05.573777 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ce9beed4-eb57-4f99-a2f2-1b92d5d3ee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-6cf38fdd-0800-4c02-8b8c-eae24b8ca0c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:05.584469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:05.585324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:05.586194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:05.586194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:05.586194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:05.586194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:05.586194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 242/496] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:05.586474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf38fdd-0800-4c02-8b8c-eae24b8ca0c2 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:05.645582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:05.653296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 255/497] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:05.654585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9beed4-eb57-4f99-a2f2-1b92d5d3ee32 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:05.696760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf38fdd-0800-4c02-8b8c-eae24b8ca0c2 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:05.733176 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52f43e-20ac-4d28-a336-7f2f8bbddf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:05.734164 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52f43e-20ac-4d28-a336-7f2f8bbddf74 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.734649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52f43e-20ac-4d28-a336-7f2f8bbddf74 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.735115 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52f43e-20ac-4d28-a336-7f2f8bbddf74 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.736288 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52f43e-20ac-4d28-a336-7f2f8bbddf74 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:05.736783 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52f43e-20ac-4d28-a336-7f2f8bbddf74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:05.738411 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52f43e-20ac-4d28-a336-7f2f8bbddf74 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:05.764830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9beed4-eb57-4f99-a2f2-1b92d5d3ee32 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:05.779151 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 243/498] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jun 17 07:07:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:05.780035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52f43e-20ac-4d28-a336-7f2f8bbddf74 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:05.785628 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:05.810025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d5e61-bb4e-4537-ad04-4e8d7fdc5e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:05.810695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d5e61-bb4e-4537-ad04-4e8d7fdc5e55 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.810988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d5e61-bb4e-4537-ad04-4e8d7fdc5e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.811267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d5e61-bb4e-4537-ad04-4e8d7fdc5e55 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.812045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d5e61-bb4e-4537-ad04-4e8d7fdc5e55 None admin] RBAC: Adding query filter params (name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:05.812378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d5e61-bb4e-4537-ad04-4e8d7fdc5e55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:05.813751 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d5e61-bb4e-4537-ad04-4e8d7fdc5e55 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:05.827142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 256/499] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:05.828010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d5e61-bb4e-4537-ad04-4e8d7fdc5e55 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:05.883743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52f43e-20ac-4d28-a336-7f2f8bbddf74 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:05.894453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:05.918040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a71e0-27aa-4a6e-8337-547bae73e295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:05.918698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a71e0-27aa-4a6e-8337-547bae73e295 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.918982 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a71e0-27aa-4a6e-8337-547bae73e295 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.919258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a71e0-27aa-4a6e-8337-547bae73e295 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.920283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a71e0-27aa-4a6e-8337-547bae73e295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:05.920588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a71e0-27aa-4a6e-8337-547bae73e295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:05.921966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a71e0-27aa-4a6e-8337-547bae73e295 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:05.925588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d5e61-bb4e-4537-ad04-4e8d7fdc5e55 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:05.933072 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 244/500] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:05.934467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a71e0-27aa-4a6e-8337-547bae73e295 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:05.958932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f2348-0329-4078-bd4a-84eb2d97cd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:05.959721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f2348-0329-4078-bd4a-84eb2d97cd16 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.960002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f2348-0329-4078-bd4a-84eb2d97cd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.960300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f2348-0329-4078-bd4a-84eb2d97cd16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.970666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f2348-0329-4078-bd4a-84eb2d97cd16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-c24f2348-0329-4078-bd4a-84eb2d97cd16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:05.978268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:05.979164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:05.979988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 257/501] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:05.980705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b07cd-1a21-4f8f-8727-61577015d16c None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:05.981028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b07cd-1a21-4f8f-8727-61577015d16c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:05.981271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b07cd-1a21-4f8f-8727-61577015d16c None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:05.982435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 258/502] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:05.983117 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b07cd-1a21-4f8f-8727-61577015d16c None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:06.018580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:06.039072 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a71e0-27aa-4a6e-8337-547bae73e295 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:06.072573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d58eaa-8229-4f1a-80e6-7fec54a63981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:06.073216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d58eaa-8229-4f1a-80e6-7fec54a63981 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:06.073494 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d58eaa-8229-4f1a-80e6-7fec54a63981 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:06.073771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d58eaa-8229-4f1a-80e6-7fec54a63981 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:06.081789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b07cd-1a21-4f8f-8727-61577015d16c None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:06.111271 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:06.114553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec0d5d-68f6-4e9b-8336-d15e76834bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:06.115318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec0d5d-68f6-4e9b-8336-d15e76834bef None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:06.115604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec0d5d-68f6-4e9b-8336-d15e76834bef None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:06.115875 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec0d5d-68f6-4e9b-8336-d15e76834bef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:06.126823 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec0d5d-68f6-4e9b-8336-d15e76834bef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:06.128293 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec0d5d-68f6-4e9b-8336-d15e76834bef None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:06.140421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d58eaa-8229-4f1a-80e6-7fec54a63981 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:06.141791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d58eaa-8229-4f1a-80e6-7fec54a63981 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-08ec0d5d-68f6-4e9b-8336-d15e76834bef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:06.145437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:06.146420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:06.147304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:06.147304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:06.147304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:06.147304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:06.147304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 259/503] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:06.147576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec0d5d-68f6-4e9b-8336-d15e76834bef None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:06.235980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:06.239978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 245/504] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:07:06.240847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d58eaa-8229-4f1a-80e6-7fec54a63981 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:06.249066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec0d5d-68f6-4e9b-8336-d15e76834bef None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:06.281441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292b9f2-fa11-46f3-9e01-0ff5b1ff0b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:06.282221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292b9f2-fa11-46f3-9e01-0ff5b1ff0b0f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:06.282507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292b9f2-fa11-46f3-9e01-0ff5b1ff0b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:06.282785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292b9f2-fa11-46f3-9e01-0ff5b1ff0b0f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:06.289982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292b9f2-fa11-46f3-9e01-0ff5b1ff0b0f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:06.291411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292b9f2-fa11-46f3-9e01-0ff5b1ff0b0f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-4292b9f2-fa11-46f3-9e01-0ff5b1ff0b0f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:06.307116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:06.308032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:06.309035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:06.309035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:06.309035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:06.309035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:06.309035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:06.309035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:06.309035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:06.309035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:06.309035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 17 07:07:06.309035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:06.309601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:06.309673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 260/505] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:06.309944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31301f5-d48a-4dcc-9dd9-3764cfee0bad None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:06.310231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31301f5-d48a-4dcc-9dd9-3764cfee0bad None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:06.310506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31301f5-d48a-4dcc-9dd9-3764cfee0bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:06.326734 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f31301f5-d48a-4dcc-9dd9-3764cfee0bad None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:06.327921 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f31301f5-d48a-4dcc-9dd9-3764cfee0bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:06.342373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d58eaa-8229-4f1a-80e6-7fec54a63981 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:06.375084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3d81f-4d0c-45be-b252-20d78d6df807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:06.375746 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3d81f-4d0c-45be-b252-20d78d6df807 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:06.376036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3d81f-4d0c-45be-b252-20d78d6df807 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:06.376349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3d81f-4d0c-45be-b252-20d78d6df807 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:06.377102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3d81f-4d0c-45be-b252-20d78d6df807 None admin] RBAC: Adding query filter params (name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:06.377415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3d81f-4d0c-45be-b252-20d78d6df807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:06.378782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3d81f-4d0c-45be-b252-20d78d6df807 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:06.390856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 246/506] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:06.390856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3d81f-4d0c-45be-b252-20d78d6df807 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:06.397384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 261/507] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:06.400938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31301f5-d48a-4dcc-9dd9-3764cfee0bad None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:06.479911 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:06.494044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3d81f-4d0c-45be-b252-20d78d6df807 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:06.503458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31301f5-d48a-4dcc-9dd9-3764cfee0bad None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:06.521997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:06.528285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20184048-1ba1-4960-9100-9c4554f98c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:06.528945 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20184048-1ba1-4960-9100-9c4554f98c70 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:06.529231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20184048-1ba1-4960-9100-9c4554f98c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:06.529514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20184048-1ba1-4960-9100-9c4554f98c70 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:06.530229 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20184048-1ba1-4960-9100-9c4554f98c70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:06.530537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20184048-1ba1-4960-9100-9c4554f98c70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:06.531897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20184048-1ba1-4960-9100-9c4554f98c70 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:06.537065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041970ca-fee1-49f7-b91d-bbfe34041d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:06.537858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041970ca-fee1-49f7-b91d-bbfe34041d06 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:06.538145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041970ca-fee1-49f7-b91d-bbfe34041d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:06.538431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041970ca-fee1-49f7-b91d-bbfe34041d06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:06.544031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 247/508] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:06.544874 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20184048-1ba1-4960-9100-9c4554f98c70 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:06.549730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041970ca-fee1-49f7-b91d-bbfe34041d06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:06.551185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041970ca-fee1-49f7-b91d-bbfe34041d06 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-041970ca-fee1-49f7-b91d-bbfe34041d06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:06.569345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:06.570209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:06.571038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:06.571038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:06.571038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:06.571038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:06.571038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 262/509] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:06.571310 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041970ca-fee1-49f7-b91d-bbfe34041d06 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:06.645519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20184048-1ba1-4960-9100-9c4554f98c70 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:06.671484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041970ca-fee1-49f7-b91d-bbfe34041d06 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:06.679846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79897a85-7a0f-4351-a1a5-fa6c423925c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:06.680544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79897a85-7a0f-4351-a1a5-fa6c423925c3 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:06.680833 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79897a85-7a0f-4351-a1a5-fa6c423925c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:06.681113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79897a85-7a0f-4351-a1a5-fa6c423925c3 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:06.681950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79897a85-7a0f-4351-a1a5-fa6c423925c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:06.682264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79897a85-7a0f-4351-a1a5-fa6c423925c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:06.683708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79897a85-7a0f-4351-a1a5-fa6c423925c3 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:06.699668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:06.701069 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 248/510] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jun 17 07:07:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:06.701777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79897a85-7a0f-4351-a1a5-fa6c423925c3 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:06.708562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ee03e-f81b-47f9-ac19-b3900e738306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:06.709430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ee03e-f81b-47f9-ac19-b3900e738306 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:06.709713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ee03e-f81b-47f9-ac19-b3900e738306 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:06.709986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ee03e-f81b-47f9-ac19-b3900e738306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:06.720542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ee03e-f81b-47f9-ac19-b3900e738306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:06.721976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ee03e-f81b-47f9-ac19-b3900e738306 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-b73ee03e-f81b-47f9-ac19-b3900e738306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:06.739579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:06.740619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:06.741850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:06.741850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:06.741850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:06.741850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:06.741850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 263/511] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:06.741850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ee03e-f81b-47f9-ac19-b3900e738306 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:06.757087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:06.806966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79897a85-7a0f-4351-a1a5-fa6c423925c3 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:06.836849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ee03e-f81b-47f9-ac19-b3900e738306 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:06.839348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab77e0-989f-4d59-98da-e12cc0951772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:06.840679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab77e0-989f-4d59-98da-e12cc0951772 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:06.840965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab77e0-989f-4d59-98da-e12cc0951772 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:06.841240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab77e0-989f-4d59-98da-e12cc0951772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:06.851772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab77e0-989f-4d59-98da-e12cc0951772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:06.853327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab77e0-989f-4d59-98da-e12cc0951772 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:06.870082 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d92e50-55e9-4ee7-9bb8-97fcb05cb40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-cfab77e0-989f-4d59-98da-e12cc0951772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:06.870359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:06.871756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:06.872649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:06.872649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:06.872649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:06.872649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:06.872856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d92e50-55e9-4ee7-9bb8-97fcb05cb40d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:06.872856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d92e50-55e9-4ee7-9bb8-97fcb05cb40d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:06.872994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 249/512] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:06.873060 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d92e50-55e9-4ee7-9bb8-97fcb05cb40d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:06.873363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab77e0-989f-4d59-98da-e12cc0951772 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:06.883301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d92e50-55e9-4ee7-9bb8-97fcb05cb40d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:06.884826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d92e50-55e9-4ee7-9bb8-97fcb05cb40d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-45d92e50-55e9-4ee7-9bb8-97fcb05cb40d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:06.899350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:06.900256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:06.901562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:06.901562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:06.901562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:06.901562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:06.901562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:06.901562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:06.901562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:06.901562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:06.901562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 07:07:06.901562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:06.901562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 264/513] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:06.902579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d92e50-55e9-4ee7-9bb8-97fcb05cb40d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:06.944228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:06.979144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab77e0-989f-4d59-98da-e12cc0951772 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:06.981306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:07.000328 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d92e50-55e9-4ee7-9bb8-97fcb05cb40d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:07.014102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e70cee-894b-49d2-a900-5164d40ec08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:07.014834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e70cee-894b-49d2-a900-5164d40ec08a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:07.015192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e70cee-894b-49d2-a900-5164d40ec08a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:07.015543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e70cee-894b-49d2-a900-5164d40ec08a None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:07.016308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e70cee-894b-49d2-a900-5164d40ec08a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:07.016622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e70cee-894b-49d2-a900-5164d40ec08a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:07.017990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e70cee-894b-49d2-a900-5164d40ec08a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:07.029197 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 250/514] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:07.029973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e70cee-894b-49d2-a900-5164d40ec08a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:07.033713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09040392-b711-44ee-a07a-370ce4dd4a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:07.034177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040392-b711-44ee-a07a-370ce4dd4a14 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:07.034464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040392-b711-44ee-a07a-370ce4dd4a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:07.034746 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040392-b711-44ee-a07a-370ce4dd4a14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:07.104467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09040392-b711-44ee-a07a-370ce4dd4a14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:07.106451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09040392-b711-44ee-a07a-370ce4dd4a14 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:07.126984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e70cee-894b-49d2-a900-5164d40ec08a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:07.146355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:07.161090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7caef-96fd-4b56-91bf-6912b1f76568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:07.161679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7caef-96fd-4b56-91bf-6912b1f76568 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:07.161959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7caef-96fd-4b56-91bf-6912b1f76568 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:07.162233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7caef-96fd-4b56-91bf-6912b1f76568 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:07.162944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7caef-96fd-4b56-91bf-6912b1f76568 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:07.163250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7caef-96fd-4b56-91bf-6912b1f76568 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:07.164694 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7caef-96fd-4b56-91bf-6912b1f76568 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:07.176472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 251/515] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:07.177250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7caef-96fd-4b56-91bf-6912b1f76568 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:07.200815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:07.206378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 265/516] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:07:07.207228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09040392-b711-44ee-a07a-370ce4dd4a14 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:07.278319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7caef-96fd-4b56-91bf-6912b1f76568 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:07.305237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09040392-b711-44ee-a07a-370ce4dd4a14 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:07.310380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059098c-dff9-40e3-a545-e981e4df68c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:07.311029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059098c-dff9-40e3-a545-e981e4df68c0 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:07.311316 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059098c-dff9-40e3-a545-e981e4df68c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:07.311590 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059098c-dff9-40e3-a545-e981e4df68c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:07.312331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059098c-dff9-40e3-a545-e981e4df68c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:07.312633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059098c-dff9-40e3-a545-e981e4df68c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:07.314006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059098c-dff9-40e3-a545-e981e4df68c0 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:07.325192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 252/517] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:07.326050 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059098c-dff9-40e3-a545-e981e4df68c0 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:07.339097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7aa686-6346-43ed-8819-b1e7215daf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:07.339819 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aa686-6346-43ed-8819-b1e7215daf3d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:07.340683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aa686-6346-43ed-8819-b1e7215daf3d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:07.340964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aa686-6346-43ed-8819-b1e7215daf3d None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:07.341798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7aa686-6346-43ed-8819-b1e7215daf3d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:07.342104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7aa686-6346-43ed-8819-b1e7215daf3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:07.343646 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7aa686-6346-43ed-8819-b1e7215daf3d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:07.377259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:07.381237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 266/518] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jun 17 07:07:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:07.382311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7aa686-6346-43ed-8819-b1e7215daf3d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:07.410306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:07.428804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059098c-dff9-40e3-a545-e981e4df68c0 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:07.461923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd82fa5-24f0-4c3e-82b0-a4ecbd986de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:07.462704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82fa5-24f0-4c3e-82b0-a4ecbd986de3 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:07.462989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82fa5-24f0-4c3e-82b0-a4ecbd986de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:07.463259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82fa5-24f0-4c3e-82b0-a4ecbd986de3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:07.469825 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd82fa5-24f0-4c3e-82b0-a4ecbd986de3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:07.471275 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd82fa5-24f0-4c3e-82b0-a4ecbd986de3 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:07.483206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7aa686-6346-43ed-8819-b1e7215daf3d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-9bd82fa5-24f0-4c3e-82b0-a4ecbd986de3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:07.485562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:07.486467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:07.487340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:07.487340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:07.487340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:07.487340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:07.487340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:07.487340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:07.487340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:07.487340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:07.488463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 253/519] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:07.489306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd82fa5-24f0-4c3e-82b0-a4ecbd986de3 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:07.516822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887d00f-fa78-478c-8b34-2224d6c80e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:07.517622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887d00f-fa78-478c-8b34-2224d6c80e36 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:07.517905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887d00f-fa78-478c-8b34-2224d6c80e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:07.518181 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887d00f-fa78-478c-8b34-2224d6c80e36 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:07.525203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887d00f-fa78-478c-8b34-2224d6c80e36 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:07.526903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887d00f-fa78-478c-8b34-2224d6c80e36 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-7887d00f-fa78-478c-8b34-2224d6c80e36 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:07.541607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:07.542500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:07.543477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:07.543477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:07.543477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:07.543477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:07.543477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:07.543477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:07.543477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:07.543477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:07.543477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 07:07:07.543477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:07.543477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 267/520] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:07.544048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887d00f-fa78-478c-8b34-2224d6c80e36 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:07.591250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd82fa5-24f0-4c3e-82b0-a4ecbd986de3 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:07.602730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:07.624315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04f162-a979-452c-b2c2-78e7b40654d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:07.625106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04f162-a979-452c-b2c2-78e7b40654d3 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:07.625390 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04f162-a979-452c-b2c2-78e7b40654d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:07.625671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04f162-a979-452c-b2c2-78e7b40654d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:07.635306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04f162-a979-452c-b2c2-78e7b40654d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:07.636806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04f162-a979-452c-b2c2-78e7b40654d3 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:07.639169 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:07.643830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887d00f-fa78-478c-8b34-2224d6c80e36 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-4a04f162-a979-452c-b2c2-78e7b40654d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:07.656226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:07.657084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:07.657933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:07.657933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:07.657933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:07.657933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:07.657933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 254/521] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:07.658259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04f162-a979-452c-b2c2-78e7b40654d3 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:07.681048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec3267-e167-4876-a1f9-4a4fc404b94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:07.681709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec3267-e167-4876-a1f9-4a4fc404b94e None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:07.681991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec3267-e167-4876-a1f9-4a4fc404b94e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:07.682288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec3267-e167-4876-a1f9-4a4fc404b94e None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:07.682977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec3267-e167-4876-a1f9-4a4fc404b94e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:07.684434 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec3267-e167-4876-a1f9-4a4fc404b94e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:07.704124 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 268/522] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:07:07 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:07.704933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec3267-e167-4876-a1f9-4a4fc404b94e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:07.779458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04f162-a979-452c-b2c2-78e7b40654d3 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:07.819599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f3a78-c643-48ce-939d-4037fa7f855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:07.820700 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f3a78-c643-48ce-939d-4037fa7f855b None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:07.820918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec3267-e167-4876-a1f9-4a4fc404b94e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:07.821367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f3a78-c643-48ce-939d-4037fa7f855b None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:07.821812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f3a78-c643-48ce-939d-4037fa7f855b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:07.839401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:07.860188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc22a0-f0e8-430d-b654-bf9fed62c29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:07.860891 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc22a0-f0e8-430d-b654-bf9fed62c29e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:07.861181 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc22a0-f0e8-430d-b654-bf9fed62c29e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:07.861461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc22a0-f0e8-430d-b654-bf9fed62c29e None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:07.863211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc22a0-f0e8-430d-b654-bf9fed62c29e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:07.863211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc22a0-f0e8-430d-b654-bf9fed62c29e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:07.863999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc22a0-f0e8-430d-b654-bf9fed62c29e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:07.884626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 269/523] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:07.885458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc22a0-f0e8-430d-b654-bf9fed62c29e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:07.899665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:07.910449 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f3a78-c643-48ce-939d-4037fa7f855b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:07.912148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f3a78-c643-48ce-939d-4037fa7f855b None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:07.990452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc22a0-f0e8-430d-b654-bf9fed62c29e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:08.020638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 255/524] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:07:08.025198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f3a78-c643-48ce-939d-4037fa7f855b None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:08.025430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cba60-2b4c-40b3-9ec4-c623cbc4130a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:08.026101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cba60-2b4c-40b3-9ec4-c623cbc4130a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:08.026648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cba60-2b4c-40b3-9ec4-c623cbc4130a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:08.026926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cba60-2b4c-40b3-9ec4-c623cbc4130a None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:08.027816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cba60-2b4c-40b3-9ec4-c623cbc4130a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:08.028162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cba60-2b4c-40b3-9ec4-c623cbc4130a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:08.029602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cba60-2b4c-40b3-9ec4-c623cbc4130a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:08.051951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 270/525] 10.222.0.33 () {56 vars in 1139 bytes} [Fri Jun 17 07:07:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:08.051951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cba60-2b4c-40b3-9ec4-c623cbc4130a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:08.086800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:08.123047 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:08.130842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f3a78-c643-48ce-939d-4037fa7f855b None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:08.159839 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cba60-2b4c-40b3-9ec4-c623cbc4130a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:08.167121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167e134-b8f4-415d-aff5-a891e8b962c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:08.167790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e134-b8f4-415d-aff5-a891e8b962c8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:08.168158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e134-b8f4-415d-aff5-a891e8b962c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:08.168957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e134-b8f4-415d-aff5-a891e8b962c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:08.169689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167e134-b8f4-415d-aff5-a891e8b962c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:08.170007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167e134-b8f4-415d-aff5-a891e8b962c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:08.171432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167e134-b8f4-415d-aff5-a891e8b962c8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:08.187285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 256/526] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:08.187285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167e134-b8f4-415d-aff5-a891e8b962c8 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:08.197213 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532455d5-c5c9-48ee-b11a-5cf0d02b4b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:08.197862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532455d5-c5c9-48ee-b11a-5cf0d02b4b4f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:08.198312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532455d5-c5c9-48ee-b11a-5cf0d02b4b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:08.198416 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532455d5-c5c9-48ee-b11a-5cf0d02b4b4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:08.199320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532455d5-c5c9-48ee-b11a-5cf0d02b4b4f None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=8d3baa16824f40f3a4c783e210b262f4) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:08.199585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532455d5-c5c9-48ee-b11a-5cf0d02b4b4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:08.201136 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532455d5-c5c9-48ee-b11a-5cf0d02b4b4f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:08.212098 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 271/527] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:08 2022] GET /identity/v3/role_assignments?user.id=8d3baa16824f40f3a4c783e210b262f4&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:08.212098 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532455d5-c5c9-48ee-b11a-5cf0d02b4b4f None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:08.294696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167e134-b8f4-415d-aff5-a891e8b962c8 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:08.331884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc8a6d-7e4d-46b9-9376-ed09ad2e23f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:08.332803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc8a6d-7e4d-46b9-9376-ed09ad2e23f2 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:08.333265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc8a6d-7e4d-46b9-9376-ed09ad2e23f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:08.333551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc8a6d-7e4d-46b9-9376-ed09ad2e23f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:08.348644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:08.349217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc8a6d-7e4d-46b9-9376-ed09ad2e23f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:08.350771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc8a6d-7e4d-46b9-9376-ed09ad2e23f2 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:08.356153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532455d5-c5c9-48ee-b11a-5cf0d02b4b4f None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-67bc8a6d-7e4d-46b9-9376-ed09ad2e23f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:08.374404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:08.375327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:08.376215 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:08.376215 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:08.376215 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:08.376215 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:08.376215 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 257/528] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:08.376835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc8a6d-7e4d-46b9-9376-ed09ad2e23f2 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:08.393323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:08.409593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac774c9-f69c-49da-a597-ce7e8e1e6cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:08.410249 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac774c9-f69c-49da-a597-ce7e8e1e6cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:08.410537 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac774c9-f69c-49da-a597-ce7e8e1e6cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:08.410812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac774c9-f69c-49da-a597-ce7e8e1e6cd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:08.481443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac774c9-f69c-49da-a597-ce7e8e1e6cd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:08.482901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac774c9-f69c-49da-a597-ce7e8e1e6cd8 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:08.529277 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc8a6d-7e4d-46b9-9376-ed09ad2e23f2 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:08.573558 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fbca2-f3c4-4eb7-9996-1899de20d2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:08.575154 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fbca2-f3c4-4eb7-9996-1899de20d2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:08.575626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fbca2-f3c4-4eb7-9996-1899de20d2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:08.576052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fbca2-f3c4-4eb7-9996-1899de20d2d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:08.583768 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fbca2-f3c4-4eb7-9996-1899de20d2d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:08.585562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:08.586652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fbca2-f3c4-4eb7-9996-1899de20d2d9 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:08.591227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 272/529] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:07:08.592183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac774c9-f69c-49da-a597-ce7e8e1e6cd8 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-254fbca2-f3c4-4eb7-9996-1899de20d2d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:08.604349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:08.605265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:08.606143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:08.606143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:08.606143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:08.606143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:08.606143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:08.606143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:08.606143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:08.606143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:08.608854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 258/530] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:08.609844 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fbca2-f3c4-4eb7-9996-1899de20d2d9 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:08.681720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:08.693523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac774c9-f69c-49da-a597-ce7e8e1e6cd8 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:08.721629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fbca2-f3c4-4eb7-9996-1899de20d2d9 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:08.727771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a611ff-eaad-439a-9f3d-d8b8e9721e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:08.728461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a611ff-eaad-439a-9f3d-d8b8e9721e84 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:08.728743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a611ff-eaad-439a-9f3d-d8b8e9721e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:08.729017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a611ff-eaad-439a-9f3d-d8b8e9721e84 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:08.729733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a611ff-eaad-439a-9f3d-d8b8e9721e84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:08.730037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a611ff-eaad-439a-9f3d-d8b8e9721e84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:08.732430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a611ff-eaad-439a-9f3d-d8b8e9721e84 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:08.747589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 273/531] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:08.748902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a611ff-eaad-439a-9f3d-d8b8e9721e84 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:08.760766 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9d784-63d4-4e47-ad11-9656759599fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:08.762115 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9d784-63d4-4e47-ad11-9656759599fe None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:08.762628 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9d784-63d4-4e47-ad11-9656759599fe None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:08.763119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9d784-63d4-4e47-ad11-9656759599fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:08.773994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9d784-63d4-4e47-ad11-9656759599fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:08.775745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9d784-63d4-4e47-ad11-9656759599fe None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-3ff9d784-63d4-4e47-ad11-9656759599fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:08.794749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:08.795646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:08.796553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:08.796553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:08.796553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:08.796553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:08.798755 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 259/532] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:08.799743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9d784-63d4-4e47-ad11-9656759599fe None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:08.819212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:08.877036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a611ff-eaad-439a-9f3d-d8b8e9721e84 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:08.910331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9d784-63d4-4e47-ad11-9656759599fe None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:08.914859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e08a05-9c73-47da-a1b1-4863ecfcf25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:08.915523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e08a05-9c73-47da-a1b1-4863ecfcf25e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:08.915804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e08a05-9c73-47da-a1b1-4863ecfcf25e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:08.916094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e08a05-9c73-47da-a1b1-4863ecfcf25e None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:08.916886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e08a05-9c73-47da-a1b1-4863ecfcf25e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:08.917195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e08a05-9c73-47da-a1b1-4863ecfcf25e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:08.919189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e08a05-9c73-47da-a1b1-4863ecfcf25e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:08.936188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 274/533] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:08.936464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:08.937059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e08a05-9c73-47da-a1b1-4863ecfcf25e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:08.948704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee89fdd-7a7d-403a-ac72-d12a13406244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:08.949656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee89fdd-7a7d-403a-ac72-d12a13406244 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:08.950052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee89fdd-7a7d-403a-ac72-d12a13406244 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:08.950235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee89fdd-7a7d-403a-ac72-d12a13406244 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:08.957975 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee89fdd-7a7d-403a-ac72-d12a13406244 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:08.959555 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee89fdd-7a7d-403a-ac72-d12a13406244 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-dee89fdd-7a7d-403a-ac72-d12a13406244 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:08.975086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:08.975933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:08.976810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:08.976810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:08.976810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:08.976810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:08.976810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:08.976810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:08.976810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:08.976810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:08.976810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 260/534] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:08.977940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee89fdd-7a7d-403a-ac72-d12a13406244 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:09.108107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee89fdd-7a7d-403a-ac72-d12a13406244 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:09.138281 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e08a05-9c73-47da-a1b1-4863ecfcf25e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:09.142741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb4ed8-8c7a-4ed8-aa55-65495311eb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:09.143565 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb4ed8-8c7a-4ed8-aa55-65495311eb48 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:09.144029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb4ed8-8c7a-4ed8-aa55-65495311eb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:09.144483 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb4ed8-8c7a-4ed8-aa55-65495311eb48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:09.145502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb4ed8-8c7a-4ed8-aa55-65495311eb48 None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=48586ff1269f49a3ae3d9f3e40c1b9c0) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:09.145977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb4ed8-8c7a-4ed8-aa55-65495311eb48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:09.147560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb4ed8-8c7a-4ed8-aa55-65495311eb48 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:09.155330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 261/535] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:08 2022] GET /identity/v3/role_assignments?user.id=48586ff1269f49a3ae3d9f3e40c1b9c0&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:09.156443 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb4ed8-8c7a-4ed8-aa55-65495311eb48 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:09.208454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:09.230019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:09.287165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fa06c-438e-4489-a04f-5d20cd5513aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:09.287825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fa06c-438e-4489-a04f-5d20cd5513aa None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:09.296356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fa06c-438e-4489-a04f-5d20cd5513aa None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:09.296356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fa06c-438e-4489-a04f-5d20cd5513aa None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:09.296356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833fa06c-438e-4489-a04f-5d20cd5513aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:09.296356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833fa06c-438e-4489-a04f-5d20cd5513aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:09.296356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833fa06c-438e-4489-a04f-5d20cd5513aa None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:09.317532 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb4ed8-8c7a-4ed8-aa55-65495311eb48 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:09.341810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 275/536] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:09.348200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fa06c-438e-4489-a04f-5d20cd5513aa None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:09.409377 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97838f1e-c9b7-4fb0-a7f2-3209dd521c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:09.410400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97838f1e-c9b7-4fb0-a7f2-3209dd521c08 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:09.410924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97838f1e-c9b7-4fb0-a7f2-3209dd521c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:09.411513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97838f1e-c9b7-4fb0-a7f2-3209dd521c08 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:09.412487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97838f1e-c9b7-4fb0-a7f2-3209dd521c08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:09.412940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97838f1e-c9b7-4fb0-a7f2-3209dd521c08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:09.414515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97838f1e-c9b7-4fb0-a7f2-3209dd521c08 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:09.470890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 262/537] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:09.477379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97838f1e-c9b7-4fb0-a7f2-3209dd521c08 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:09.608292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fa06c-438e-4489-a04f-5d20cd5513aa None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:09.625559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97838f1e-c9b7-4fb0-a7f2-3209dd521c08 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:09.655437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:09.665055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78858e8-b54b-46f6-bece-51d47c26534e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:09.666636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78858e8-b54b-46f6-bece-51d47c26534e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:09.667819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78858e8-b54b-46f6-bece-51d47c26534e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:09.668872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78858e8-b54b-46f6-bece-51d47c26534e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:09.669873 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78858e8-b54b-46f6-bece-51d47c26534e None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=3769534c610b448db693f7c5d6133e65) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:09.670353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78858e8-b54b-46f6-bece-51d47c26534e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:09.671960 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78858e8-b54b-46f6-bece-51d47c26534e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:09.678369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 263/538] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:09 2022] GET /identity/v3/role_assignments?user.id=3769534c610b448db693f7c5d6133e65&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:09.687236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee0885-e740-4b51-bbc2-51423dfd167e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:09.688158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee0885-e740-4b51-bbc2-51423dfd167e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:09.688452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee0885-e740-4b51-bbc2-51423dfd167e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:09.688793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee0885-e740-4b51-bbc2-51423dfd167e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:09.709583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee0885-e740-4b51-bbc2-51423dfd167e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:09.711095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee0885-e740-4b51-bbc2-51423dfd167e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-95ee0885-e740-4b51-bbc2-51423dfd167e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:09.738189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:09.739300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:09.740330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:09.740330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:09.740330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:09.740330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:09.740330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:09.740330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:09.740330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:09.740330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:09.740330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 276/539] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:09.745193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78858e8-b54b-46f6-bece-51d47c26534e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:09.908229 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78858e8-b54b-46f6-bece-51d47c26534e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:09.963123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6f94c-3473-4caa-98e1-56f7dae1eaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:09.963791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6f94c-3473-4caa-98e1-56f7dae1eaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:09.964087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6f94c-3473-4caa-98e1-56f7dae1eaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:09.964373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6f94c-3473-4caa-98e1-56f7dae1eaf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:09.965775 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6f94c-3473-4caa-98e1-56f7dae1eaf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:09.966085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6f94c-3473-4caa-98e1-56f7dae1eaf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:09.967536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6f94c-3473-4caa-98e1-56f7dae1eaf8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:09.984934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 264/540] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:09.990702 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee0885-e740-4b51-bbc2-51423dfd167e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:10.079221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:10.172972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee0885-e740-4b51-bbc2-51423dfd167e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:10.228996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4cfc9-9cbb-4ee3-a6c6-2f527b9a9c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:10.229689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4cfc9-9cbb-4ee3-a6c6-2f527b9a9c2b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:10.229973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4cfc9-9cbb-4ee3-a6c6-2f527b9a9c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:10.230251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4cfc9-9cbb-4ee3-a6c6-2f527b9a9c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:10.231643 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4cfc9-9cbb-4ee3-a6c6-2f527b9a9c2b None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=7e0797679c07454abc867cd7b30c8b56) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:10.231957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4cfc9-9cbb-4ee3-a6c6-2f527b9a9c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:10.233451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4cfc9-9cbb-4ee3-a6c6-2f527b9a9c2b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:10.240253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 277/541] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:09 2022] GET /identity/v3/role_assignments?user.id=7e0797679c07454abc867cd7b30c8b56&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:10.308386 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfaa31-f10d-4af6-99f8-ecd948632d1d None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:10.308748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfaa31-f10d-4af6-99f8-ecd948632d1d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:10.309012 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfaa31-f10d-4af6-99f8-ecd948632d1d None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:10.310643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 265/542] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:10.315921 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85309b1e-6536-4f5f-8d0b-bd8fc63f6471 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:10.316271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85309b1e-6536-4f5f-8d0b-bd8fc63f6471 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:10.316532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85309b1e-6536-4f5f-8d0b-bd8fc63f6471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:10.341421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-85309b1e-6536-4f5f-8d0b-bd8fc63f6471 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:10.342673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-85309b1e-6536-4f5f-8d0b-bd8fc63f6471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:10.446358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 278/543] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:10.481603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfaa31-f10d-4af6-99f8-ecd948632d1d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:10.521314 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:10.616779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3896b4e-dd0d-4c4c-8783-37dcd5981a24 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:10.617741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3896b4e-dd0d-4c4c-8783-37dcd5981a24 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:10.618229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3896b4e-dd0d-4c4c-8783-37dcd5981a24 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:10.620089 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 279/544] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:10.624088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfaa31-f10d-4af6-99f8-ecd948632d1d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:10.626140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa599e4b-9730-4bd2-b400-b518bc2ab649 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:10.626649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa599e4b-9730-4bd2-b400-b518bc2ab649 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:10.627133 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa599e4b-9730-4bd2-b400-b518bc2ab649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:10.645591 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-aa599e4b-9730-4bd2-b400-b518bc2ab649 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:10.647777 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-aa599e4b-9730-4bd2-b400-b518bc2ab649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:10.661055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd52b94-4be6-48cb-a4a4-915e9210c448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:10.661858 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd52b94-4be6-48cb-a4a4-915e9210c448 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:10.662201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd52b94-4be6-48cb-a4a4-915e9210c448 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:10.662485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd52b94-4be6-48cb-a4a4-915e9210c448 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:10.673515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd52b94-4be6-48cb-a4a4-915e9210c448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:10.679799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-7bd52b94-4be6-48cb-a4a4-915e9210c448 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:10.681237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:10.682150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:10.684016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 266/545] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:10.689880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd52b94-4be6-48cb-a4a4-915e9210c448 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:10.765863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 280/546] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:07:10.807291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa599e4b-9730-4bd2-b400-b518bc2ab649 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:10.822832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:10.851805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd52b94-4be6-48cb-a4a4-915e9210c448 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:10.921938 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f27d5-9981-4e04-967b-d34601062e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:10.922621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f27d5-9981-4e04-967b-d34601062e0c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:10.922908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f27d5-9981-4e04-967b-d34601062e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:10.923192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f27d5-9981-4e04-967b-d34601062e0c None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:10.923949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f27d5-9981-4e04-967b-d34601062e0c None admin] RBAC: Adding query filter params (name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:10.924284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f27d5-9981-4e04-967b-d34601062e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:10.925681 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f27d5-9981-4e04-967b-d34601062e0c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:10.937490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 267/547] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:10.942491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f27d5-9981-4e04-967b-d34601062e0c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:10.964103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa599e4b-9730-4bd2-b400-b518bc2ab649 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:11.032435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e4b71-544f-496a-b05d-ce852c20b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:11.033162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e4b71-544f-496a-b05d-ce852c20b22d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:11.033450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e4b71-544f-496a-b05d-ce852c20b22d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:11.033730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e4b71-544f-496a-b05d-ce852c20b22d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:11.057502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e4b71-544f-496a-b05d-ce852c20b22d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-3a6e4b71-544f-496a-b05d-ce852c20b22d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:11.081453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:11.082405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:11.083656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 281/548] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:11.092683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e4b71-544f-496a-b05d-ce852c20b22d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:11.136412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:11.137184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f27d5-9981-4e04-967b-d34601062e0c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:11.175082 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8b10f-ee32-471b-9919-0bcb3cd54c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:11.175883 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8b10f-ee32-471b-9919-0bcb3cd54c21 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:11.176208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8b10f-ee32-471b-9919-0bcb3cd54c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:11.176498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8b10f-ee32-471b-9919-0bcb3cd54c21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:11.203625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8b10f-ee32-471b-9919-0bcb3cd54c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:11.207336 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8b10f-ee32-471b-9919-0bcb3cd54c21 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:11.225499 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-92c8b10f-ee32-471b-9919-0bcb3cd54c21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:11.243910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:11.245222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:11.246682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:11.246682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:11.246682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:11.246682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:11.246682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 268/549] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:11.252716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8b10f-ee32-471b-9919-0bcb3cd54c21 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:11.275734 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e4b71-544f-496a-b05d-ce852c20b22d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:11.344888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cc0db-6efa-4461-9e8b-1e66a1e206d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:11.345480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cc0db-6efa-4461-9e8b-1e66a1e206d9 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:11.345769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cc0db-6efa-4461-9e8b-1e66a1e206d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:11.346049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cc0db-6efa-4461-9e8b-1e66a1e206d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:11.346796 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cc0db-6efa-4461-9e8b-1e66a1e206d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:11.347106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cc0db-6efa-4461-9e8b-1e66a1e206d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:11.354858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cc0db-6efa-4461-9e8b-1e66a1e206d9 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:11.372320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 282/550] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:11.377410 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cc0db-6efa-4461-9e8b-1e66a1e206d9 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:11.466280 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8b10f-ee32-471b-9919-0bcb3cd54c21 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:11.544991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:11.551167 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cc0db-6efa-4461-9e8b-1e66a1e206d9 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:11.560886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d31e05-c7dd-4fba-8ab1-dc8e456fa524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:11.561593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d31e05-c7dd-4fba-8ab1-dc8e456fa524 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:11.562171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d31e05-c7dd-4fba-8ab1-dc8e456fa524 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:11.562171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d31e05-c7dd-4fba-8ab1-dc8e456fa524 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:11.562987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d31e05-c7dd-4fba-8ab1-dc8e456fa524 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:11.563308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d31e05-c7dd-4fba-8ab1-dc8e456fa524 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:11.564752 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d31e05-c7dd-4fba-8ab1-dc8e456fa524 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:11.586807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 269/551] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:11.593100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d31e05-c7dd-4fba-8ab1-dc8e456fa524 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:11.612278 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452fbf0-6a14-4da8-abb6-3cd4b640bd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:11.612988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452fbf0-6a14-4da8-abb6-3cd4b640bd2b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:11.613282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452fbf0-6a14-4da8-abb6-3cd4b640bd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:11.613556 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452fbf0-6a14-4da8-abb6-3cd4b640bd2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:11.624127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452fbf0-6a14-4da8-abb6-3cd4b640bd2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:11.625597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452fbf0-6a14-4da8-abb6-3cd4b640bd2b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-f452fbf0-6a14-4da8-abb6-3cd4b640bd2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:11.647788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:11.648837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:11.650438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:11.650438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:11.650438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:11.650438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:11.650438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 283/552] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:11.650725 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8150eed-655b-496b-b1a2-e8604a6448b7 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:11.650950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8150eed-655b-496b-b1a2-e8604a6448b7 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:11.651475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8150eed-655b-496b-b1a2-e8604a6448b7 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:11.653415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 284/553] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:11.654647 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8150eed-655b-496b-b1a2-e8604a6448b7 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:11.669625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:11.707452 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d31e05-c7dd-4fba-8ab1-dc8e456fa524 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:11.743392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d998e-15fa-4b16-ac4a-f8e8daf986dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:11.744337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d998e-15fa-4b16-ac4a-f8e8daf986dc None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:11.744623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d998e-15fa-4b16-ac4a-f8e8daf986dc None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:11.744897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d998e-15fa-4b16-ac4a-f8e8daf986dc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:11.755821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d998e-15fa-4b16-ac4a-f8e8daf986dc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:11.757474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d998e-15fa-4b16-ac4a-f8e8daf986dc None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-6a7d998e-15fa-4b16-ac4a-f8e8daf986dc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:11.773018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:11.773984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:11.774991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:11.774991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:11.774991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:11.774991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:11.774991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:11.774991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:11.774991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:11.774991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:11.774991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 07:07:11.774991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:11.779550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 270/554] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:07:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:11.781104 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ff0eb-f45f-4479-a6b8-a06e7365d36f None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:11.781104 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ff0eb-f45f-4479-a6b8-a06e7365d36f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:11.782200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ff0eb-f45f-4479-a6b8-a06e7365d36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:11.786906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8150eed-655b-496b-b1a2-e8604a6448b7 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:11.799524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c03ff0eb-f45f-4479-a6b8-a06e7365d36f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:11.800884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c03ff0eb-f45f-4479-a6b8-a06e7365d36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:11.837848 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10d8c7-96b3-42a7-a39e-e9ab9fae886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:11.838926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10d8c7-96b3-42a7-a39e-e9ab9fae886e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:11.839451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10d8c7-96b3-42a7-a39e-e9ab9fae886e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:11.840543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10d8c7-96b3-42a7-a39e-e9ab9fae886e None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:11.841496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10d8c7-96b3-42a7-a39e-e9ab9fae886e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:11.842051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10d8c7-96b3-42a7-a39e-e9ab9fae886e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:11.844243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10d8c7-96b3-42a7-a39e-e9ab9fae886e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:11.848133 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:11.861947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 285/555] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:11.863443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10d8c7-96b3-42a7-a39e-e9ab9fae886e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:11.872964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 271/556] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:11.873907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ff0eb-f45f-4479-a6b8-a06e7365d36f None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:11.931634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:11.985110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10d8c7-96b3-42a7-a39e-e9ab9fae886e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:12.000278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ff0eb-f45f-4479-a6b8-a06e7365d36f None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:12.020776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b273cd5-7907-4ef4-9645-4b35c8fc8aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:12.021470 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b273cd5-7907-4ef4-9645-4b35c8fc8aad None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.021751 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b273cd5-7907-4ef4-9645-4b35c8fc8aad None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.022022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b273cd5-7907-4ef4-9645-4b35c8fc8aad None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.022857 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b273cd5-7907-4ef4-9645-4b35c8fc8aad None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:12.023163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b273cd5-7907-4ef4-9645-4b35c8fc8aad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:12.024623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b273cd5-7907-4ef4-9645-4b35c8fc8aad None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:12.045666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901320a-7fcc-45c3-adb9-a53b0ce36716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:12.046814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901320a-7fcc-45c3-adb9-a53b0ce36716 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.047318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901320a-7fcc-45c3-adb9-a53b0ce36716 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.047508 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 286/557] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jun 17 07:07:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:12.047975 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901320a-7fcc-45c3-adb9-a53b0ce36716 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.048373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b273cd5-7907-4ef4-9645-4b35c8fc8aad None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:12.056708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901320a-7fcc-45c3-adb9-a53b0ce36716 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:12.058910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901320a-7fcc-45c3-adb9-a53b0ce36716 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-f901320a-7fcc-45c3-adb9-a53b0ce36716 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:12.074813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:12.076289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:12.077216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:12.077216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:12.077216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:12.077216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:12.077216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:12.077216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:12.077216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:12.077216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:12.077216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:07:12.077216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:12.077216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 272/558] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jun 17 07:07:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:12.077818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901320a-7fcc-45c3-adb9-a53b0ce36716 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:12.121690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:12.181002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b273cd5-7907-4ef4-9645-4b35c8fc8aad None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:12.204813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901320a-7fcc-45c3-adb9-a53b0ce36716 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:12.216940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:12.229418 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256a97f-5fea-4b6e-a5c1-430c8951deec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:12.230786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256a97f-5fea-4b6e-a5c1-430c8951deec None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.231357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256a97f-5fea-4b6e-a5c1-430c8951deec None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.231357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256a97f-5fea-4b6e-a5c1-430c8951deec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.241933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256a97f-5fea-4b6e-a5c1-430c8951deec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:12.242306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7f0dc-7bfd-4686-8e4a-1e816d3bb5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:12.243167 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7f0dc-7bfd-4686-8e4a-1e816d3bb5cc None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.243572 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7f0dc-7bfd-4686-8e4a-1e816d3bb5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.243880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7f0dc-7bfd-4686-8e4a-1e816d3bb5cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-8256a97f-5fea-4b6e-a5c1-430c8951deec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:12.250425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:12.251505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:12.252406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 287/559] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:12.253100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256a97f-5fea-4b6e-a5c1-430c8951deec None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:12.322011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7f0dc-7bfd-4686-8e4a-1e816d3bb5cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:12.323462 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7f0dc-7bfd-4686-8e4a-1e816d3bb5cc None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:12.363727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256a97f-5fea-4b6e-a5c1-430c8951deec None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:12.378967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:12.407803 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abefa2c-157e-4d88-b45b-0b01d4a1cad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:12.408503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abefa2c-157e-4d88-b45b-0b01d4a1cad2 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.408778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abefa2c-157e-4d88-b45b-0b01d4a1cad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.409207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abefa2c-157e-4d88-b45b-0b01d4a1cad2 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.410458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abefa2c-157e-4d88-b45b-0b01d4a1cad2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:12.411025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abefa2c-157e-4d88-b45b-0b01d4a1cad2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:12.412671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abefa2c-157e-4d88-b45b-0b01d4a1cad2 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:12.430140 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 273/560] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:07:12.439864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7041f-381a-418c-9d57-3f04b30d0ecf None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.439864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7041f-381a-418c-9d57-3f04b30d0ecf None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.439864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7041f-381a-418c-9d57-3f04b30d0ecf None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.439864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 274/561] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:12.439864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7041f-381a-418c-9d57-3f04b30d0ecf None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:12.440311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 288/562] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jun 17 07:07:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:12.440311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abefa2c-157e-4d88-b45b-0b01d4a1cad2 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:12.452408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:12.546159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7041f-381a-418c-9d57-3f04b30d0ecf None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:12.550976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abefa2c-157e-4d88-b45b-0b01d4a1cad2 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:12.582057 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f08b4-6529-4c4a-aecb-7c15ba0971d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:12.582730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f08b4-6529-4c4a-aecb-7c15ba0971d6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.583018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f08b4-6529-4c4a-aecb-7c15ba0971d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.583301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f08b4-6529-4c4a-aecb-7c15ba0971d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.584050 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f08b4-6529-4c4a-aecb-7c15ba0971d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:12.584390 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f08b4-6529-4c4a-aecb-7c15ba0971d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:12.585781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f08b4-6529-4c4a-aecb-7c15ba0971d6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:12.588344 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d660c3f-25e4-4dbd-b293-bfe463841bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:12.589208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d660c3f-25e4-4dbd-b293-bfe463841bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.589635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d660c3f-25e4-4dbd-b293-bfe463841bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.589985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d660c3f-25e4-4dbd-b293-bfe463841bf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.598354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 275/563] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:12.598354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f08b4-6529-4c4a-aecb-7c15ba0971d6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:12.602786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d660c3f-25e4-4dbd-b293-bfe463841bf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:12.604371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d660c3f-25e4-4dbd-b293-bfe463841bf0 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-9d660c3f-25e4-4dbd-b293-bfe463841bf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:12.622723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:12.623846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:12.625401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:12.625401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:12.625401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:12.625401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:12.625833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 289/564] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:12.627075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348654fa-b00b-4bce-ab88-1329919314a0 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.627366 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348654fa-b00b-4bce-ab88-1329919314a0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.627650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348654fa-b00b-4bce-ab88-1329919314a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.646202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-348654fa-b00b-4bce-ab88-1329919314a0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:12.647376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-348654fa-b00b-4bce-ab88-1329919314a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:12.667292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:12.683181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:12.716473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f08b4-6529-4c4a-aecb-7c15ba0971d6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:12.721765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 290/565] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:12.722606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348654fa-b00b-4bce-ab88-1329919314a0 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:12.752624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64486d9-4e35-4869-921f-8bd1c14351f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:12.753451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64486d9-4e35-4869-921f-8bd1c14351f9 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.753739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64486d9-4e35-4869-921f-8bd1c14351f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.754017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64486d9-4e35-4869-921f-8bd1c14351f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.764332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64486d9-4e35-4869-921f-8bd1c14351f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:12.765781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64486d9-4e35-4869-921f-8bd1c14351f9 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-d64486d9-4e35-4869-921f-8bd1c14351f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:12.783004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:12.784055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:12.785195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:12.785195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:12.785195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:12.785195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:12.785195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 276/566] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:12.785195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64486d9-4e35-4869-921f-8bd1c14351f9 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:12.827963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348654fa-b00b-4bce-ab88-1329919314a0 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:12.862000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c9a0e-dc51-4a6f-9dc7-22924a014bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:12.862789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9a0e-dc51-4a6f-9dc7-22924a014bbb None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.863076 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9a0e-dc51-4a6f-9dc7-22924a014bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.863354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9a0e-dc51-4a6f-9dc7-22924a014bbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.873214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c9a0e-dc51-4a6f-9dc7-22924a014bbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:12.874658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c9a0e-dc51-4a6f-9dc7-22924a014bbb None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:12.881726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64486d9-4e35-4869-921f-8bd1c14351f9 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-9d2c9a0e-dc51-4a6f-9dc7-22924a014bbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:12.891890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:12.892982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:12.894071 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:12.894071 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:12.894071 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:12.894071 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:12.894071 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 291/567] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:12.894424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fc573-6598-4f2e-b78b-0d55c085dfda None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.894555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fc573-6598-4f2e-b78b-0d55c085dfda None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.894893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fc573-6598-4f2e-b78b-0d55c085dfda None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.896419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 292/568] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:12.897081 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fc573-6598-4f2e-b78b-0d55c085dfda None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:12.906641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:12.909184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:12.921076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f32e2-12f5-4d1d-a014-560b93543973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:12.921734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f32e2-12f5-4d1d-a014-560b93543973 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:12.922024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f32e2-12f5-4d1d-a014-560b93543973 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:12.922301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f32e2-12f5-4d1d-a014-560b93543973 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:12.923022 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f32e2-12f5-4d1d-a014-560b93543973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:12.923330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f32e2-12f5-4d1d-a014-560b93543973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:12.924726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f32e2-12f5-4d1d-a014-560b93543973 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:12.941980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 277/569] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:12.941980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f32e2-12f5-4d1d-a014-560b93543973 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.001153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fc573-6598-4f2e-b78b-0d55c085dfda None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.035032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f32e2-12f5-4d1d-a014-560b93543973 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.036624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c6993-4583-40ff-a8b8-7b25a2f50f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.037415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c6993-4583-40ff-a8b8-7b25a2f50f0d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.037705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c6993-4583-40ff-a8b8-7b25a2f50f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.037989 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c6993-4583-40ff-a8b8-7b25a2f50f0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.047865 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c6993-4583-40ff-a8b8-7b25a2f50f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-4f7c6993-4583-40ff-a8b8-7b25a2f50f0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:13.055529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:13.056566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:13.057433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 293/570] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:13.058116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c6993-4583-40ff-a8b8-7b25a2f50f0d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.069456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466eadab-ab31-4240-a339-ab0e12c1de60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.070112 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466eadab-ab31-4240-a339-ab0e12c1de60 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.070392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466eadab-ab31-4240-a339-ab0e12c1de60 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.070663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466eadab-ab31-4240-a339-ab0e12c1de60 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.071374 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466eadab-ab31-4240-a339-ab0e12c1de60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:13.071678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466eadab-ab31-4240-a339-ab0e12c1de60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:13.073073 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466eadab-ab31-4240-a339-ab0e12c1de60 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:13.084503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 278/571] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:13.085639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223f08-0e63-4405-b8d2-fb65cd81208e None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.085932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223f08-0e63-4405-b8d2-fb65cd81208e None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.086208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223f08-0e63-4405-b8d2-fb65cd81208e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.101643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-79223f08-0e63-4405-b8d2-fb65cd81208e None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:13.102816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-79223f08-0e63-4405-b8d2-fb65cd81208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:13.129291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:13.143033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:13.163291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c6993-4583-40ff-a8b8-7b25a2f50f0d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.170574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 279/572] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:13.171399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223f08-0e63-4405-b8d2-fb65cd81208e None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.198206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7099b07-444d-40ff-b303-21fc75a9cd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.198894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7099b07-444d-40ff-b303-21fc75a9cd6b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.199184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7099b07-444d-40ff-b303-21fc75a9cd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.199488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7099b07-444d-40ff-b303-21fc75a9cd6b None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.200216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7099b07-444d-40ff-b303-21fc75a9cd6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:13.200549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7099b07-444d-40ff-b303-21fc75a9cd6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:13.201905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7099b07-444d-40ff-b303-21fc75a9cd6b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:13.212989 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 294/573] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:13.213771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7099b07-444d-40ff-b303-21fc75a9cd6b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.273980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223f08-0e63-4405-b8d2-fb65cd81208e None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.307464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c81eaa-bda2-497b-a843-e3ffade50144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.308864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c81eaa-bda2-497b-a843-e3ffade50144 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.309159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c81eaa-bda2-497b-a843-e3ffade50144 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.309448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c81eaa-bda2-497b-a843-e3ffade50144 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.312017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7099b07-444d-40ff-b303-21fc75a9cd6b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.316099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c81eaa-bda2-497b-a843-e3ffade50144 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:13.317567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c81eaa-bda2-497b-a843-e3ffade50144 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-47c81eaa-bda2-497b-a843-e3ffade50144 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:13.331174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:13.332254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:13.333244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:13.333244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:13.333244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:13.333244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:13.333244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:13.333244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:13.333244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:13.333244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:13.336231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 280/574] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:13.336421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:13.337093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c81eaa-bda2-497b-a843-e3ffade50144 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.345798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc36456-d397-4d24-b7ac-c3f897dd528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.346466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc36456-d397-4d24-b7ac-c3f897dd528d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.346754 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc36456-d397-4d24-b7ac-c3f897dd528d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.347031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc36456-d397-4d24-b7ac-c3f897dd528d None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.347773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc36456-d397-4d24-b7ac-c3f897dd528d None admin] RBAC: Adding query filter params (name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:13.348105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc36456-d397-4d24-b7ac-c3f897dd528d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:13.349473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc36456-d397-4d24-b7ac-c3f897dd528d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:13.357408 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 295/575] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:13.358182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc36456-d397-4d24-b7ac-c3f897dd528d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.374269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:13.440793 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c81eaa-bda2-497b-a843-e3ffade50144 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.462490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc36456-d397-4d24-b7ac-c3f897dd528d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.473855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d1400-da73-42bd-9cc7-45d55579a175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.474645 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d1400-da73-42bd-9cc7-45d55579a175 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.474934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d1400-da73-42bd-9cc7-45d55579a175 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.475206 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d1400-da73-42bd-9cc7-45d55579a175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.482274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d1400-da73-42bd-9cc7-45d55579a175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:13.483718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d1400-da73-42bd-9cc7-45d55579a175 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:13.495018 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f861d-f808-46b8-a93c-df5028e37dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.495822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f861d-f808-46b8-a93c-df5028e37dda None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.496130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f861d-f808-46b8-a93c-df5028e37dda None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.496404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f861d-f808-46b8-a93c-df5028e37dda None admin] PATH_INFO: `/v3/roles/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-fa0d1400-da73-42bd-9cc7-45d55579a175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:13.497641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:13.498571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:13.499503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:13.499503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:13.499503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:13.499503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:13.499503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:13.499503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:13.499503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:13.499503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:13.499503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 281/576] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:13.499993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d1400-da73-42bd-9cc7-45d55579a175 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.507993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f861d-f808-46b8-a93c-df5028e37dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-989f861d-f808-46b8-a93c-df5028e37dda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:13.515774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:13.516885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 296/577] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:13.517852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f861d-f808-46b8-a93c-df5028e37dda None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.566825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:13.594217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:13.601046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d1400-da73-42bd-9cc7-45d55579a175 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.617094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f861d-f808-46b8-a93c-df5028e37dda None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.634789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491808ad-67bb-44d6-a048-83908ba745fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.635584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491808ad-67bb-44d6-a048-83908ba745fc None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.635874 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491808ad-67bb-44d6-a048-83908ba745fc None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.636192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491808ad-67bb-44d6-a048-83908ba745fc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.643155 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491808ad-67bb-44d6-a048-83908ba745fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:13.644638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491808ad-67bb-44d6-a048-83908ba745fc None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:13.649903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742965e6-aafa-4a80-9d52-eeaa04415649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.650557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742965e6-aafa-4a80-9d52-eeaa04415649 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.650844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742965e6-aafa-4a80-9d52-eeaa04415649 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.651378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742965e6-aafa-4a80-9d52-eeaa04415649 None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.652676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742965e6-aafa-4a80-9d52-eeaa04415649 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:13.652988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742965e6-aafa-4a80-9d52-eeaa04415649 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:13.654450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742965e6-aafa-4a80-9d52-eeaa04415649 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-491808ad-67bb-44d6-a048-83908ba745fc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:13.659889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:13.660981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:13.666866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:13.666866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:13.666866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:13.666866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:13.666866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:13.666866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:13.666866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:13.666866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:13.666866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 17 07:07:13.666866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:13.666866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 282/578] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:13.666866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491808ad-67bb-44d6-a048-83908ba745fc None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.667723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 297/579] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:13.668157 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742965e6-aafa-4a80-9d52-eeaa04415649 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.761156 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491808ad-67bb-44d6-a048-83908ba745fc None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.766594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742965e6-aafa-4a80-9d52-eeaa04415649 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.776255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:13.795361 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc5086-90df-4d87-94e8-105024aacf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.796182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc5086-90df-4d87-94e8-105024aacf70 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.796479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc5086-90df-4d87-94e8-105024aacf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.796760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc5086-90df-4d87-94e8-105024aacf70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.801980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f4be4-b9e2-44de-b241-a7af248484be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.802637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f4be4-b9e2-44de-b241-a7af248484be None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.802924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f4be4-b9e2-44de-b241-a7af248484be None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.803204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f4be4-b9e2-44de-b241-a7af248484be None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.803984 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f4be4-b9e2-44de-b241-a7af248484be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:13.804371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f4be4-b9e2-44de-b241-a7af248484be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:13.805798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f4be4-b9e2-44de-b241-a7af248484be None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:13.808184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc5086-90df-4d87-94e8-105024aacf70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:13.809651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc5086-90df-4d87-94e8-105024aacf70 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:13.817082 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 298/580] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:13.817868 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f4be4-b9e2-44de-b241-a7af248484be None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-cfbc5086-90df-4d87-94e8-105024aacf70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:13.829459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:13.830515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:13.831707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:13.831707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:13.831707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:13.831707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:13.831707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 283/581] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:13.831707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc5086-90df-4d87-94e8-105024aacf70 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.841687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:13.917474 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f4be4-b9e2-44de-b241-a7af248484be None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.931022 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc5086-90df-4d87-94e8-105024aacf70 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:13.951208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98b773-dda5-4ade-b2bc-b7dd149cce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.951868 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98b773-dda5-4ade-b2bc-b7dd149cce7a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.952185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98b773-dda5-4ade-b2bc-b7dd149cce7a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.952465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98b773-dda5-4ade-b2bc-b7dd149cce7a None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.953203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98b773-dda5-4ade-b2bc-b7dd149cce7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:13.953507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98b773-dda5-4ade-b2bc-b7dd149cce7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:13.954876 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98b773-dda5-4ade-b2bc-b7dd149cce7a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:13.962620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 299/582] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:13.963394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98b773-dda5-4ade-b2bc-b7dd149cce7a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.968339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7e4e5-cd86-4ec9-838c-b422ef767284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:13.969004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7e4e5-cd86-4ec9-838c-b422ef767284 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:13.969288 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7e4e5-cd86-4ec9-838c-b422ef767284 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:13.969567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7e4e5-cd86-4ec9-838c-b422ef767284 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:13.970402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7e4e5-cd86-4ec9-838c-b422ef767284 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:13.970713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7e4e5-cd86-4ec9-838c-b422ef767284 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:13.972156 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7e4e5-cd86-4ec9-838c-b422ef767284 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:13.988920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 284/583] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jun 17 07:07:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:13.989683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7e4e5-cd86-4ec9-838c-b422ef767284 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:13.997047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:14.051092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:14.070159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98b773-dda5-4ade-b2bc-b7dd149cce7a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:14.088315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7e4e5-cd86-4ec9-838c-b422ef767284 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:14.103882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57289d-6bb6-4c79-8dc3-fce87cbae600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:14.104563 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57289d-6bb6-4c79-8dc3-fce87cbae600 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:14.104850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57289d-6bb6-4c79-8dc3-fce87cbae600 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:14.105154 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57289d-6bb6-4c79-8dc3-fce87cbae600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:14.105983 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57289d-6bb6-4c79-8dc3-fce87cbae600 None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=004e5909f9834cfea890063bb0083d7c) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:14.106303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57289d-6bb6-4c79-8dc3-fce87cbae600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:14.107736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57289d-6bb6-4c79-8dc3-fce87cbae600 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:14.113202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 300/584] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:13 2022] GET /identity/v3/role_assignments?user.id=004e5909f9834cfea890063bb0083d7c&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:14.113977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57289d-6bb6-4c79-8dc3-fce87cbae600 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:14.133931 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d893e16-1a1e-4282-b69d-a45a89b66589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:14.134849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d893e16-1a1e-4282-b69d-a45a89b66589 None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:14.135131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d893e16-1a1e-4282-b69d-a45a89b66589 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:14.135408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d893e16-1a1e-4282-b69d-a45a89b66589 None admin] PATH_INFO: `/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/8d3baa16824f40f3a4c783e210b262f4/roles/2a15b4ff19204ac5916a0099ec8660c6` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:14.154513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d893e16-1a1e-4282-b69d-a45a89b66589 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4ec9b09302d4c918ef638d794468ba3, user_id=8d3baa16824f40f3a4c783e210b262f4, role_id=2a15b4ff19204ac5916a0099ec8660c6)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:14.155983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d893e16-1a1e-4282-b69d-a45a89b66589 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:14.171686 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 285/585] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:07:13 2022] PUT /identity/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/8d3baa16824f40f3a4c783e210b262f4/roles/2a15b4ff19204ac5916a0099ec8660c6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:07:14.173072 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d893e16-1a1e-4282-b69d-a45a89b66589 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:14.233998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:14.244354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57289d-6bb6-4c79-8dc3-fce87cbae600 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:14.290838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269796c-2f07-4bb9-a8b1-546b4299ac86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:14.291523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269796c-2f07-4bb9-a8b1-546b4299ac86 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:14.291850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269796c-2f07-4bb9-a8b1-546b4299ac86 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:14.292124 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269796c-2f07-4bb9-a8b1-546b4299ac86 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:14.293450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269796c-2f07-4bb9-a8b1-546b4299ac86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:14.294528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269796c-2f07-4bb9-a8b1-546b4299ac86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:14.296109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269796c-2f07-4bb9-a8b1-546b4299ac86 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:14.312286 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 301/586] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:14.312286 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269796c-2f07-4bb9-a8b1-546b4299ac86 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:14.336390 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d893e16-1a1e-4282-b69d-a45a89b66589 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:14.354474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:14.384924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57356bde-5f92-4795-92f7-2cc53aa7c8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:14.384924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57356bde-5f92-4795-92f7-2cc53aa7c8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:14.384924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57356bde-5f92-4795-92f7-2cc53aa7c8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:14.384924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57356bde-5f92-4795-92f7-2cc53aa7c8e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:14.397914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57356bde-5f92-4795-92f7-2cc53aa7c8e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:14.399413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57356bde-5f92-4795-92f7-2cc53aa7c8e5 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-57356bde-5f92-4795-92f7-2cc53aa7c8e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:14.426706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:14.427669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:14.428672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:14.428672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:14.428672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:14.428672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:14.428672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 286/587] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:14.434501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57356bde-5f92-4795-92f7-2cc53aa7c8e5 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:14.448209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269796c-2f07-4bb9-a8b1-546b4299ac86 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:14.492248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6493469-6c78-403a-b2e0-5c189b257431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:14.493321 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6493469-6c78-403a-b2e0-5c189b257431 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:14.493616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6493469-6c78-403a-b2e0-5c189b257431 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:14.493898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6493469-6c78-403a-b2e0-5c189b257431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:14.507671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6493469-6c78-403a-b2e0-5c189b257431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:14.509890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6493469-6c78-403a-b2e0-5c189b257431 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-b6493469-6c78-403a-b2e0-5c189b257431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:14.534694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:14.535650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:14.536592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:14.536592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:14.536592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:14.536592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:14.540150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 302/588] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:14.541225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6493469-6c78-403a-b2e0-5c189b257431 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:14.564793 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57356bde-5f92-4795-92f7-2cc53aa7c8e5 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:14.592441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:14.617382 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780442d-39c6-43aa-b916-475d1dc81c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:14.618587 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780442d-39c6-43aa-b916-475d1dc81c55 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:14.619290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780442d-39c6-43aa-b916-475d1dc81c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:14.619681 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780442d-39c6-43aa-b916-475d1dc81c55 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:14.629630 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780442d-39c6-43aa-b916-475d1dc81c55 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:14.631220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780442d-39c6-43aa-b916-475d1dc81c55 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-a780442d-39c6-43aa-b916-475d1dc81c55 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:14.651595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:14.653188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:14.654222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:14.654222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:14.654222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:14.654222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:14.654222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:14.654222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:14.654222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:14.654222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:14.654222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 07:07:14.654222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:14.654222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 287/589] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:14.654811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780442d-39c6-43aa-b916-475d1dc81c55 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:14.666655 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6493469-6c78-403a-b2e0-5c189b257431 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:14.690150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:14.706325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54b37b-dc7c-4550-9c7b-d889d8da1b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:14.707322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54b37b-dc7c-4550-9c7b-d889d8da1b80 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:14.707817 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54b37b-dc7c-4550-9c7b-d889d8da1b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:14.708348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54b37b-dc7c-4550-9c7b-d889d8da1b80 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:14.709444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54b37b-dc7c-4550-9c7b-d889d8da1b80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:14.709959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54b37b-dc7c-4550-9c7b-d889d8da1b80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:14.711772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54b37b-dc7c-4550-9c7b-d889d8da1b80 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:14.732335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 303/590] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:14.733614 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54b37b-dc7c-4550-9c7b-d889d8da1b80 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:14.781051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780442d-39c6-43aa-b916-475d1dc81c55 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:14.817311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76eacf-3704-4d0d-bc67-224fed7e704e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:14.818302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76eacf-3704-4d0d-bc67-224fed7e704e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:14.818808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76eacf-3704-4d0d-bc67-224fed7e704e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:14.819289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76eacf-3704-4d0d-bc67-224fed7e704e None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:14.820263 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76eacf-3704-4d0d-bc67-224fed7e704e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:14.820930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76eacf-3704-4d0d-bc67-224fed7e704e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:14.822511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76eacf-3704-4d0d-bc67-224fed7e704e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:14.832482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:14.835438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 288/591] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:14.836478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76eacf-3704-4d0d-bc67-224fed7e704e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:14.859382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54b37b-dc7c-4550-9c7b-d889d8da1b80 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:14.902492 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12110a59-4b73-4115-b10d-34d72257e401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:14.903177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12110a59-4b73-4115-b10d-34d72257e401 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:14.903466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12110a59-4b73-4115-b10d-34d72257e401 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:14.903754 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12110a59-4b73-4115-b10d-34d72257e401 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:14.904954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12110a59-4b73-4115-b10d-34d72257e401 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:14.905414 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12110a59-4b73-4115-b10d-34d72257e401 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:14.907171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12110a59-4b73-4115-b10d-34d72257e401 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:14.930382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 304/592] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jun 17 07:07:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:14.931582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12110a59-4b73-4115-b10d-34d72257e401 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:14.933660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:14.951226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76eacf-3704-4d0d-bc67-224fed7e704e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:14.995317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6b0c7-fd53-4801-8549-495af9a414bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:14.996214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6b0c7-fd53-4801-8549-495af9a414bf None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:14.996549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6b0c7-fd53-4801-8549-495af9a414bf None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:14.996837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6b0c7-fd53-4801-8549-495af9a414bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:15.008357 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6b0c7-fd53-4801-8549-495af9a414bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:15.010519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6b0c7-fd53-4801-8549-495af9a414bf None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-5cc6b0c7-fd53-4801-8549-495af9a414bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:15.030005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:15.030996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:15.031966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:15.031966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:15.031966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:15.031966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:15.031966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:15.031966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:15.031966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:15.031966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:15.031966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 07:07:15.031966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:15.031966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 289/593] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:15.031966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6b0c7-fd53-4801-8549-495af9a414bf None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:15.053196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12110a59-4b73-4115-b10d-34d72257e401 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:15.078952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:15.093118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96891cb6-20f0-4405-8baf-378065e0efd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:15.094168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96891cb6-20f0-4405-8baf-378065e0efd8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:15.094640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96891cb6-20f0-4405-8baf-378065e0efd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:15.095100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96891cb6-20f0-4405-8baf-378065e0efd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:15.102352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96891cb6-20f0-4405-8baf-378065e0efd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:15.104090 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96891cb6-20f0-4405-8baf-378065e0efd8 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-96891cb6-20f0-4405-8baf-378065e0efd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:15.118986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:15.119882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:15.121393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:15.121393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:15.121393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:15.121393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:15.121393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:15.121393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:15.121393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:15.121393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:15.123869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 305/594] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:15.124879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96891cb6-20f0-4405-8baf-378065e0efd8 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:15.162773 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6b0c7-fd53-4801-8549-495af9a414bf None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:15.198384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20719b-37f4-43bf-8e7f-d5ee485ec204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:15.199608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20719b-37f4-43bf-8e7f-d5ee485ec204 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:15.200242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20719b-37f4-43bf-8e7f-d5ee485ec204 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:15.200744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20719b-37f4-43bf-8e7f-d5ee485ec204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:15.211836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20719b-37f4-43bf-8e7f-d5ee485ec204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:15.213527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20719b-37f4-43bf-8e7f-d5ee485ec204 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:15.221003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-1a20719b-37f4-43bf-8e7f-d5ee485ec204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:15.234716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:15.235642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:15.236932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:15.236932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:15.236932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:15.236932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:15.238988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 290/595] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:15.240076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20719b-37f4-43bf-8e7f-d5ee485ec204 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:15.257335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96891cb6-20f0-4405-8baf-378065e0efd8 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:15.295253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764f9f7-875c-464a-bc74-9aa21368c648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:15.296304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764f9f7-875c-464a-bc74-9aa21368c648 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:15.296799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764f9f7-875c-464a-bc74-9aa21368c648 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:15.297350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764f9f7-875c-464a-bc74-9aa21368c648 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:15.298437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764f9f7-875c-464a-bc74-9aa21368c648 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:15.298984 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764f9f7-875c-464a-bc74-9aa21368c648 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:15.300662 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764f9f7-875c-464a-bc74-9aa21368c648 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:15.326917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 306/596] 10.222.0.33 () {56 vars in 1139 bytes} [Fri Jun 17 07:07:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:15.340167 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764f9f7-875c-464a-bc74-9aa21368c648 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:15.368760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20719b-37f4-43bf-8e7f-d5ee485ec204 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:15.395434 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:15.413864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51227c36-e5bd-4b6b-bf34-5e2bd8fe0442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:15.414854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51227c36-e5bd-4b6b-bf34-5e2bd8fe0442 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:15.415214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51227c36-e5bd-4b6b-bf34-5e2bd8fe0442 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:15.415676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51227c36-e5bd-4b6b-bf34-5e2bd8fe0442 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:15.416971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51227c36-e5bd-4b6b-bf34-5e2bd8fe0442 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:15.417360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51227c36-e5bd-4b6b-bf34-5e2bd8fe0442 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:15.420023 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51227c36-e5bd-4b6b-bf34-5e2bd8fe0442 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:15.440553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 291/597] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:15.440553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51227c36-e5bd-4b6b-bf34-5e2bd8fe0442 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:15.468313 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764f9f7-875c-464a-bc74-9aa21368c648 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:15.509056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:15.511030 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8ec41-9c00-4ded-a5fc-c024381353e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:15.512971 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8ec41-9c00-4ded-a5fc-c024381353e5 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:15.513343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8ec41-9c00-4ded-a5fc-c024381353e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:15.513679 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8ec41-9c00-4ded-a5fc-c024381353e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:15.514789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8ec41-9c00-4ded-a5fc-c024381353e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:15.515207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8ec41-9c00-4ded-a5fc-c024381353e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:15.517250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8ec41-9c00-4ded-a5fc-c024381353e5 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:15.532176 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 307/598] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:15.533482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8ec41-9c00-4ded-a5fc-c024381353e5 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:15.558124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51227c36-e5bd-4b6b-bf34-5e2bd8fe0442 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:15.601724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb9f85-c59b-473f-89a0-afefd56e867a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:15.602847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb9f85-c59b-473f-89a0-afefd56e867a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:15.603231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb9f85-c59b-473f-89a0-afefd56e867a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:15.603846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb9f85-c59b-473f-89a0-afefd56e867a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:15.615403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb9f85-c59b-473f-89a0-afefd56e867a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:15.617206 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb9f85-c59b-473f-89a0-afefd56e867a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-3ddb9f85-c59b-473f-89a0-afefd56e867a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:15.637003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:15.637904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:15.638811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:15.638811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:15.638811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:15.638811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:15.641112 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 292/599] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:15.642193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb9f85-c59b-473f-89a0-afefd56e867a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:15.651647 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8ec41-9c00-4ded-a5fc-c024381353e5 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:15.653712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:15.689461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29370c66-421c-43f6-91f0-06de337e3366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:15.690357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29370c66-421c-43f6-91f0-06de337e3366 None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:15.690644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29370c66-421c-43f6-91f0-06de337e3366 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:15.690920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29370c66-421c-43f6-91f0-06de337e3366 None admin] PATH_INFO: `/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/48586ff1269f49a3ae3d9f3e40c1b9c0/roles/2a15b4ff19204ac5916a0099ec8660c6` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:15.709758 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29370c66-421c-43f6-91f0-06de337e3366 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4ec9b09302d4c918ef638d794468ba3, user_id=48586ff1269f49a3ae3d9f3e40c1b9c0, role_id=2a15b4ff19204ac5916a0099ec8660c6)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:15.711364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29370c66-421c-43f6-91f0-06de337e3366 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:15.736139 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 308/600] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:07:15 2022] PUT /identity/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/48586ff1269f49a3ae3d9f3e40c1b9c0/roles/2a15b4ff19204ac5916a0099ec8660c6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:07:15.736139 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29370c66-421c-43f6-91f0-06de337e3366 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:15.781568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb9f85-c59b-473f-89a0-afefd56e867a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:15.804385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:15.860232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29370c66-421c-43f6-91f0-06de337e3366 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:15.878592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5e129-5886-42ec-b4c5-e602835a7096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:15.879208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5e129-5886-42ec-b4c5-e602835a7096 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:15.879592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5e129-5886-42ec-b4c5-e602835a7096 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:15.879982 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5e129-5886-42ec-b4c5-e602835a7096 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:15.901439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5e129-5886-42ec-b4c5-e602835a7096 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:15.903207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5e129-5886-42ec-b4c5-e602835a7096 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:15.903645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c828-32e6-4877-891e-25a01739526d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:15.904614 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c828-32e6-4877-891e-25a01739526d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:15.905038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c828-32e6-4877-891e-25a01739526d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:15.905471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c828-32e6-4877-891e-25a01739526d None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:15.906407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992c828-32e6-4877-891e-25a01739526d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:15.906852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992c828-32e6-4877-891e-25a01739526d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:15.908577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992c828-32e6-4877-891e-25a01739526d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:15.922965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 309/601] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:15.928212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c828-32e6-4877-891e-25a01739526d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-40a5e129-5886-42ec-b4c5-e602835a7096 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:15.943921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:15.945106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:15.946249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:15.946249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:15.946249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:15.946249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:15.946249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:15.946249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:15.946249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:15.946249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:15.948210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 293/602] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:15.954534 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5e129-5886-42ec-b4c5-e602835a7096 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:15.964788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:16.078181 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c828-32e6-4877-891e-25a01739526d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:16.088706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5e129-5886-42ec-b4c5-e602835a7096 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:16.118378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bae64a-c437-4a64-a6cf-86e4af99f98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:16.119246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bae64a-c437-4a64-a6cf-86e4af99f98e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:16.119635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bae64a-c437-4a64-a6cf-86e4af99f98e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:16.120015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bae64a-c437-4a64-a6cf-86e4af99f98e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:16.128154 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bae64a-c437-4a64-a6cf-86e4af99f98e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:16.130004 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bae64a-c437-4a64-a6cf-86e4af99f98e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:16.130779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fb47e-1efe-4c5e-98ba-62b801119fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:16.131617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fb47e-1efe-4c5e-98ba-62b801119fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:16.132253 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fb47e-1efe-4c5e-98ba-62b801119fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:16.132706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fb47e-1efe-4c5e-98ba-62b801119fc8 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:16.133976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fb47e-1efe-4c5e-98ba-62b801119fc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:16.134442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fb47e-1efe-4c5e-98ba-62b801119fc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:16.136031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fb47e-1efe-4c5e-98ba-62b801119fc8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-82bae64a-c437-4a64-a6cf-86e4af99f98e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:16.146542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:16.147644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:16.148800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:16.148800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:16.148800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:16.148800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:16.148800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:16.148800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:16.148800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:16.148800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:16.150617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 310/603] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:16.153076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 294/604] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:16.157201 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bae64a-c437-4a64-a6cf-86e4af99f98e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:16.158750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fb47e-1efe-4c5e-98ba-62b801119fc8 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:16.222900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:16.311470 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:16.365171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bae64a-c437-4a64-a6cf-86e4af99f98e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:16.391319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fb47e-1efe-4c5e-98ba-62b801119fc8 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:16.410618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35728244-be0c-4e5e-a922-f1741f7752b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:16.411421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35728244-be0c-4e5e-a922-f1741f7752b6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:16.411782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35728244-be0c-4e5e-a922-f1741f7752b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:16.412195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35728244-be0c-4e5e-a922-f1741f7752b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:16.413062 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35728244-be0c-4e5e-a922-f1741f7752b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:16.413439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35728244-be0c-4e5e-a922-f1741f7752b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:16.414934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35728244-be0c-4e5e-a922-f1741f7752b6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:16.428113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 311/605] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:16.428526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bbd13-ee3a-4673-b834-19bfd37279ba None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:16.428822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bbd13-ee3a-4673-b834-19bfd37279ba None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:16.429107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bbd13-ee3a-4673-b834-19bfd37279ba None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:16.430456 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 312/606] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:16.433814 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b64335-2f17-4cbf-a5ef-236a7ace3b6b None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:16.434102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b64335-2f17-4cbf-a5ef-236a7ace3b6b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:16.434381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b64335-2f17-4cbf-a5ef-236a7ace3b6b None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:16.435645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 313/607] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:16.436367 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b64335-2f17-4cbf-a5ef-236a7ace3b6b None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:16.440139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cfd9f-3278-499a-8dd5-3a14e0c98386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:16.442162 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cfd9f-3278-499a-8dd5-3a14e0c98386 None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:16.442675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cfd9f-3278-499a-8dd5-3a14e0c98386 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:16.444262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cfd9f-3278-499a-8dd5-3a14e0c98386 None admin] PATH_INFO: `/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/3769534c610b448db693f7c5d6133e65/roles/2a15b4ff19204ac5916a0099ec8660c6` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:16.459080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cfd9f-3278-499a-8dd5-3a14e0c98386 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4ec9b09302d4c918ef638d794468ba3, user_id=3769534c610b448db693f7c5d6133e65, role_id=2a15b4ff19204ac5916a0099ec8660c6)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:16.460564 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cfd9f-3278-499a-8dd5-3a14e0c98386 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:16.476010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 295/608] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:07:16 2022] PUT /identity/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/3769534c610b448db693f7c5d6133e65/roles/2a15b4ff19204ac5916a0099ec8660c6 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:07:16.477643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb885ff-f58a-4b91-b1f4-275a2345aa31 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:16.478005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb885ff-f58a-4b91-b1f4-275a2345aa31 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:16.478360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb885ff-f58a-4b91-b1f4-275a2345aa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:16.500816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-adb885ff-f58a-4b91-b1f4-275a2345aa31 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:16.502409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-adb885ff-f58a-4b91-b1f4-275a2345aa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:16.564395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b64335-2f17-4cbf-a5ef-236a7ace3b6b None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:16.594463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 296/609] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:16.596941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b5fdf-0832-41aa-9fb8-d550138a3beb None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:16.597744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b5fdf-0832-41aa-9fb8-d550138a3beb None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:16.598881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:16.599154 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b5fdf-0832-41aa-9fb8-d550138a3beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:16.616275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0f7e0-a929-4c32-b9fa-7576a0a429ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:16.616275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0f7e0-a929-4c32-b9fa-7576a0a429ff None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:16.616275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0f7e0-a929-4c32-b9fa-7576a0a429ff None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:16.616275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0f7e0-a929-4c32-b9fa-7576a0a429ff None admin] PATH_INFO: `/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/7e0797679c07454abc867cd7b30c8b56/roles/2a15b4ff19204ac5916a0099ec8660c6` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:16.626627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-d34b5fdf-0832-41aa-9fb8-d550138a3beb None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:16.628289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d34b5fdf-0832-41aa-9fb8-d550138a3beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:16.632443 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:16.635535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0f7e0-a929-4c32-b9fa-7576a0a429ff None admin] RBAC: Authorizing `identity:create_grant(project_id=a4ec9b09302d4c918ef638d794468ba3, user_id=7e0797679c07454abc867cd7b30c8b56, role_id=2a15b4ff19204ac5916a0099ec8660c6)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:16.637849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0f7e0-a929-4c32-b9fa-7576a0a429ff None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:16.655544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 314/610] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:07:16 2022] PUT /identity/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/7e0797679c07454abc867cd7b30c8b56/roles/2a15b4ff19204ac5916a0099ec8660c6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:07:16.656788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0f7e0-a929-4c32-b9fa-7576a0a429ff None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:16.739017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 297/611] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:16.769873 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b5fdf-0832-41aa-9fb8-d550138a3beb None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:16.784195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0f7e0-a929-4c32-b9fa-7576a0a429ff None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:16.846159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d7cf0-50e3-40c2-b646-dedc7fd34bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:16.847391 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d7cf0-50e3-40c2-b646-dedc7fd34bdb None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:16.847883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d7cf0-50e3-40c2-b646-dedc7fd34bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:16.848427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d7cf0-50e3-40c2-b646-dedc7fd34bdb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:16.871218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d7cf0-50e3-40c2-b646-dedc7fd34bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-336d7cf0-50e3-40c2-b646-dedc7fd34bdb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:16.880094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:16.881022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:16.881863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:16.881863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:16.881863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:16.881863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:16.881863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:16.881863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:16.881863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:16.881863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:16.881863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:16.881863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:16.881863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:16.881863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:16.884628 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 315/612] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:16.890737 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d7cf0-50e3-40c2-b646-dedc7fd34bdb None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:16.917151 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b5fdf-0832-41aa-9fb8-d550138a3beb None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:16.954932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:16.956235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1958e8-8785-44ac-8bac-4f3578065489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:16.957398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1958e8-8785-44ac-8bac-4f3578065489 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:16.957720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1958e8-8785-44ac-8bac-4f3578065489 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:16.957998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1958e8-8785-44ac-8bac-4f3578065489 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:16.967672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:16.971139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1958e8-8785-44ac-8bac-4f3578065489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-3d1958e8-8785-44ac-8bac-4f3578065489 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:16.979716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:16.981217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:16.982058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 298/613] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:16.988180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1958e8-8785-44ac-8bac-4f3578065489 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:17.034048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d7cf0-50e3-40c2-b646-dedc7fd34bdb None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:17.097020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657572ee-eced-45a0-8cda-fdd5bc6edadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:17.097976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657572ee-eced-45a0-8cda-fdd5bc6edadb None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:17.098434 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657572ee-eced-45a0-8cda-fdd5bc6edadb None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:17.098861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657572ee-eced-45a0-8cda-fdd5bc6edadb None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:17.099802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657572ee-eced-45a0-8cda-fdd5bc6edadb None admin] RBAC: Adding query filter params (name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:17.100296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657572ee-eced-45a0-8cda-fdd5bc6edadb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:17.103404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657572ee-eced-45a0-8cda-fdd5bc6edadb None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:17.126858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 316/614] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:17.129837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657572ee-eced-45a0-8cda-fdd5bc6edadb None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:17.152370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1958e8-8785-44ac-8bac-4f3578065489 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:17.222225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d91ddc-9e6a-4ab4-a9a2-985dbc5ee6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:17.222225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d91ddc-9e6a-4ab4-a9a2-985dbc5ee6fb None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:17.222225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d91ddc-9e6a-4ab4-a9a2-985dbc5ee6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:17.222225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d91ddc-9e6a-4ab4-a9a2-985dbc5ee6fb None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:17.223343 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d91ddc-9e6a-4ab4-a9a2-985dbc5ee6fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:17.223748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d91ddc-9e6a-4ab4-a9a2-985dbc5ee6fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:17.225242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d91ddc-9e6a-4ab4-a9a2-985dbc5ee6fb None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:17.242944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 299/615] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:17.253889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d91ddc-9e6a-4ab4-a9a2-985dbc5ee6fb None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:17.272920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657572ee-eced-45a0-8cda-fdd5bc6edadb None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:17.275287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:17.321462 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dd24b-8553-4bd3-908a-4a4e1579931b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:17.322274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dd24b-8553-4bd3-908a-4a4e1579931b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:17.322566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dd24b-8553-4bd3-908a-4a4e1579931b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:17.322846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dd24b-8553-4bd3-908a-4a4e1579931b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:17.339125 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468dd24b-8553-4bd3-908a-4a4e1579931b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:17.341153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468dd24b-8553-4bd3-908a-4a4e1579931b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:17.354255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-468dd24b-8553-4bd3-908a-4a4e1579931b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:17.361040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:17.362188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:17.363303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:17.363303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:17.363303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:17.363303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:17.363303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 317/616] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:17.367910 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dd24b-8553-4bd3-908a-4a4e1579931b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:17.435633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d91ddc-9e6a-4ab4-a9a2-985dbc5ee6fb None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:17.502685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b011d-c206-415b-9a8d-dcdea6d54d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:17.503528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b011d-c206-415b-9a8d-dcdea6d54d52 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:17.503828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b011d-c206-415b-9a8d-dcdea6d54d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:17.504129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b011d-c206-415b-9a8d-dcdea6d54d52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:17.515407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b011d-c206-415b-9a8d-dcdea6d54d52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:17.516894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b011d-c206-415b-9a8d-dcdea6d54d52 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-848b011d-c206-415b-9a8d-dcdea6d54d52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:17.535764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:17.536859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:17.537933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:17.537933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:17.537933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:17.537933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:17.537933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 300/617] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:17.546236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b011d-c206-415b-9a8d-dcdea6d54d52 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:17.547657 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dd24b-8553-4bd3-908a-4a4e1579931b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:17.601760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c77de-7c6b-417b-8e76-83d0cf67a520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:17.602437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c77de-7c6b-417b-8e76-83d0cf67a520 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:17.602721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c77de-7c6b-417b-8e76-83d0cf67a520 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:17.602999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c77de-7c6b-417b-8e76-83d0cf67a520 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:17.603721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c77de-7c6b-417b-8e76-83d0cf67a520 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:17.604028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c77de-7c6b-417b-8e76-83d0cf67a520 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:17.607134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c77de-7c6b-417b-8e76-83d0cf67a520 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:17.640112 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 318/618] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:17.640112 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c77de-7c6b-417b-8e76-83d0cf67a520 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:17.658948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:17.763424 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b011d-c206-415b-9a8d-dcdea6d54d52 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:17.792485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:17.809017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff866ea-6235-402f-ba76-0b27c6e748ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:17.809825 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff866ea-6235-402f-ba76-0b27c6e748ae None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:17.810220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff866ea-6235-402f-ba76-0b27c6e748ae None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:17.810607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff866ea-6235-402f-ba76-0b27c6e748ae None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:17.811440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff866ea-6235-402f-ba76-0b27c6e748ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:17.811862 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff866ea-6235-402f-ba76-0b27c6e748ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:17.813890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff866ea-6235-402f-ba76-0b27c6e748ae None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:17.826950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c77de-7c6b-417b-8e76-83d0cf67a520 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:17.827519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 301/619] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:17.833673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff866ea-6235-402f-ba76-0b27c6e748ae None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:17.895872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb65732-e93c-4764-a48c-45b51af8cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:17.896699 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb65732-e93c-4764-a48c-45b51af8cb8e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:17.898974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb65732-e93c-4764-a48c-45b51af8cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:17.899999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb65732-e93c-4764-a48c-45b51af8cb8e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:17.914445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb65732-e93c-4764-a48c-45b51af8cb8e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:17.915881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb65732-e93c-4764-a48c-45b51af8cb8e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-bbb65732-e93c-4764-a48c-45b51af8cb8e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:17.943248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:17.944475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:17.946046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:17.946046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:17.946046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:17.946046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:17.946046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:17.946046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:17.946046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:17.946046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:17.946046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 07:07:17.946046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:17.946603 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 319/620] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:07:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:17.951132 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb65732-e93c-4764-a48c-45b51af8cb8e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:18.043853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff866ea-6235-402f-ba76-0b27c6e748ae None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:18.075633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:18.106411 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729b92b-510d-45fb-a1ec-1281719ca69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:18.107206 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729b92b-510d-45fb-a1ec-1281719ca69d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:18.107489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729b92b-510d-45fb-a1ec-1281719ca69d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:18.107766 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729b92b-510d-45fb-a1ec-1281719ca69d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:18.117537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729b92b-510d-45fb-a1ec-1281719ca69d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:18.119003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729b92b-510d-45fb-a1ec-1281719ca69d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:18.121866 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb65732-e93c-4764-a48c-45b51af8cb8e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-5729b92b-510d-45fb-a1ec-1281719ca69d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:18.143671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:18.144716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:18.145664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:18.145664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:18.145664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:18.145664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:18.145664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:18.145664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:18.145664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:18.145664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:18.145664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:07:18.145664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:18.152710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 302/621] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jun 17 07:07:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:18.157306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729b92b-510d-45fb-a1ec-1281719ca69d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:18.186018 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab666ef-d2e1-4407-ba80-4aadc6f39dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:18.186695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab666ef-d2e1-4407-ba80-4aadc6f39dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:18.186976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab666ef-d2e1-4407-ba80-4aadc6f39dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:18.187418 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab666ef-d2e1-4407-ba80-4aadc6f39dd5 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:18.188284 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab666ef-d2e1-4407-ba80-4aadc6f39dd5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:18.188592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab666ef-d2e1-4407-ba80-4aadc6f39dd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:18.190012 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab666ef-d2e1-4407-ba80-4aadc6f39dd5 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:18.219358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 320/622] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jun 17 07:07:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:18.224321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:18.225087 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab666ef-d2e1-4407-ba80-4aadc6f39dd5 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:18.338335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729b92b-510d-45fb-a1ec-1281719ca69d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:18.387893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab666ef-d2e1-4407-ba80-4aadc6f39dd5 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:18.392572 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc607b-60f7-4df9-833d-cc361be72613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:18.393439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc607b-60f7-4df9-833d-cc361be72613 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:18.393871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc607b-60f7-4df9-833d-cc361be72613 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:18.394293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc607b-60f7-4df9-833d-cc361be72613 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:18.395259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc607b-60f7-4df9-833d-cc361be72613 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:18.395706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc607b-60f7-4df9-833d-cc361be72613 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:18.397294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc607b-60f7-4df9-833d-cc361be72613 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:18.417977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 303/623] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jun 17 07:07:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:18.419569 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a81d01-5cf4-4ccb-a3f5-50474ceb97a5 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:18.420014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a81d01-5cf4-4ccb-a3f5-50474ceb97a5 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:18.420474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a81d01-5cf4-4ccb-a3f5-50474ceb97a5 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:18.422199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 304/624] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:18.423068 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a81d01-5cf4-4ccb-a3f5-50474ceb97a5 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:18.432587 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06b810-f4c2-4db6-973e-5674b8f1a40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:18.433721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06b810-f4c2-4db6-973e-5674b8f1a40a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:18.434181 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06b810-f4c2-4db6-973e-5674b8f1a40a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:18.434640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06b810-f4c2-4db6-973e-5674b8f1a40a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:18.437092 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:18.447764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06b810-f4c2-4db6-973e-5674b8f1a40a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:18.450109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06b810-f4c2-4db6-973e-5674b8f1a40a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-3f06b810-f4c2-4db6-973e-5674b8f1a40a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:18.469750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:18.470718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:18.471602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:18.471602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:18.471602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:18.471602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:18.473970 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 321/625] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:18.475540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0c921-919f-4c50-afcf-e5cc88a6cb89 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:18.476042 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0c921-919f-4c50-afcf-e5cc88a6cb89 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:18.476539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0c921-919f-4c50-afcf-e5cc88a6cb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:18.493902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-65f0c921-919f-4c50-afcf-e5cc88a6cb89 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:18.495414 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-65f0c921-919f-4c50-afcf-e5cc88a6cb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:18.535726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:18.547516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a81d01-5cf4-4ccb-a3f5-50474ceb97a5 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:18.584943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf0546-ecb0-45d9-bc1c-ddf4caf29c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:18.586019 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf0546-ecb0-45d9-bc1c-ddf4caf29c66 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:18.586593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf0546-ecb0-45d9-bc1c-ddf4caf29c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:18.587067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf0546-ecb0-45d9-bc1c-ddf4caf29c66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:18.588150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 322/626] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:18.589877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0c921-919f-4c50-afcf-e5cc88a6cb89 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:18.671492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf0546-ecb0-45d9-bc1c-ddf4caf29c66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:18.673366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf0546-ecb0-45d9-bc1c-ddf4caf29c66 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:18.728191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0c921-919f-4c50-afcf-e5cc88a6cb89 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:18.751508 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:18.774196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04aadea-4266-4297-a20d-242b2c1e5fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:18.775236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:18.776303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04aadea-4266-4297-a20d-242b2c1e5fae None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:18.776444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04aadea-4266-4297-a20d-242b2c1e5fae None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:18.777278 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04aadea-4266-4297-a20d-242b2c1e5fae None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:18.778035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04aadea-4266-4297-a20d-242b2c1e5fae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:18.778327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04aadea-4266-4297-a20d-242b2c1e5fae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:18.779994 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04aadea-4266-4297-a20d-242b2c1e5fae None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:18.790363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 305/627] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:07:18.791648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf0546-ecb0-45d9-bc1c-ddf4caf29c66 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:18.794931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 323/628] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:18.796798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04aadea-4266-4297-a20d-242b2c1e5fae None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:18.901972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04aadea-4266-4297-a20d-242b2c1e5fae None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:18.922054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf0546-ecb0-45d9-bc1c-ddf4caf29c66 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:18.937828 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9337d-a8c6-4ffb-90fb-e9d11fe78d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:18.938956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9337d-a8c6-4ffb-90fb-e9d11fe78d44 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:18.939441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9337d-a8c6-4ffb-90fb-e9d11fe78d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:18.939919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9337d-a8c6-4ffb-90fb-e9d11fe78d44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:18.953250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9337d-a8c6-4ffb-90fb-e9d11fe78d44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:18.955000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9337d-a8c6-4ffb-90fb-e9d11fe78d44 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:18.966097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ffc9a-296e-4fd7-94fe-ff727e972170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:18.966896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ffc9a-296e-4fd7-94fe-ff727e972170 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:18.967180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ffc9a-296e-4fd7-94fe-ff727e972170 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:18.967444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ffc9a-296e-4fd7-94fe-ff727e972170 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-f5c9337d-a8c6-4ffb-90fb-e9d11fe78d44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:18.974113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:18.975075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:18.975998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:18.975998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:18.975998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:18.975998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:18.977805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ffc9a-296e-4fd7-94fe-ff727e972170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:18.978683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 324/629] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:18.980330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9337d-a8c6-4ffb-90fb-e9d11fe78d44 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-0b4ffc9a-296e-4fd7-94fe-ff727e972170 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:18.987491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:18.988500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 306/630] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:18.989745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ffc9a-296e-4fd7-94fe-ff727e972170 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:19.017609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:19.073512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:19.109034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9337d-a8c6-4ffb-90fb-e9d11fe78d44 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:19.123912 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ffc9a-296e-4fd7-94fe-ff727e972170 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:19.144212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e955e-ecc5-4891-9809-5657b259d5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:19.144995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e955e-ecc5-4891-9809-5657b259d5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.145281 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e955e-ecc5-4891-9809-5657b259d5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.145565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e955e-ecc5-4891-9809-5657b259d5d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.152805 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e955e-ecc5-4891-9809-5657b259d5d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:19.154305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e955e-ecc5-4891-9809-5657b259d5d8 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:19.155883 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575d26a-9f29-4822-9950-a6520a186adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:19.156603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575d26a-9f29-4822-9950-a6520a186adf None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.156997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575d26a-9f29-4822-9950-a6520a186adf None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.157316 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575d26a-9f29-4822-9950-a6520a186adf None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.158265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575d26a-9f29-4822-9950-a6520a186adf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:19.158583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575d26a-9f29-4822-9950-a6520a186adf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:19.160262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575d26a-9f29-4822-9950-a6520a186adf None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-e35e955e-ecc5-4891-9809-5657b259d5d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:19.169842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:19.170841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:19.171733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:19.171733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:19.171733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:19.171733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:19.171733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:19.171733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:19.171733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:19.171733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:19.171733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 325/631] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:19.172708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e955e-ecc5-4891-9809-5657b259d5d8 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:19.172781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 307/632] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:19.173946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760c2d9-ba38-41b0-97ef-c39a9b62610f None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.174237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760c2d9-ba38-41b0-97ef-c39a9b62610f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.174512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760c2d9-ba38-41b0-97ef-c39a9b62610f None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.175950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 308/633] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:19.176980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21bbb7-02ef-4b94-bf67-7f7b192837ff None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.177207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21bbb7-02ef-4b94-bf67-7f7b192837ff None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.177539 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21bbb7-02ef-4b94-bf67-7f7b192837ff None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.178761 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 309/634] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:19.179444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21bbb7-02ef-4b94-bf67-7f7b192837ff None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:19.261939 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:19.273991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e955e-ecc5-4891-9809-5657b259d5d8 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:19.280418 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21bbb7-02ef-4b94-bf67-7f7b192837ff None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:19.293870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:19.307567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d9c33-1911-4228-82b8-81851691d642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:19.308796 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d9c33-1911-4228-82b8-81851691d642 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.309080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d9c33-1911-4228-82b8-81851691d642 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.309354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d9c33-1911-4228-82b8-81851691d642 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.310088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d9c33-1911-4228-82b8-81851691d642 None admin] RBAC: Adding query filter params (name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:19.310394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d9c33-1911-4228-82b8-81851691d642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:19.311752 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d9c33-1911-4228-82b8-81851691d642 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:19.315795 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c595a6-1ca7-450e-9f25-98076449bacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:19.316475 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c595a6-1ca7-450e-9f25-98076449bacf None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.316763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c595a6-1ca7-450e-9f25-98076449bacf None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.317043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c595a6-1ca7-450e-9f25-98076449bacf None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.317824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c595a6-1ca7-450e-9f25-98076449bacf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:19.318139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c595a6-1ca7-450e-9f25-98076449bacf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:19.319466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 326/635] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:19.319565 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c595a6-1ca7-450e-9f25-98076449bacf None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:19.320266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d9c33-1911-4228-82b8-81851691d642 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:19.331558 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 310/636] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:19.332630 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d32ba-7b10-47b6-b214-d0a748c7af17 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.332924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d32ba-7b10-47b6-b214-d0a748c7af17 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.333208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d32ba-7b10-47b6-b214-d0a748c7af17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.349220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-ee6d32ba-7b10-47b6-b214-d0a748c7af17 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:19.350385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ee6d32ba-7b10-47b6-b214-d0a748c7af17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:19.418078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 311/637] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:19.419348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8aca6e-1279-4095-a612-db67115479be None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.419668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8aca6e-1279-4095-a612-db67115479be None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.419927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8aca6e-1279-4095-a612-db67115479be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.423052 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d9c33-1911-4228-82b8-81851691d642 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:19.438196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-ba8aca6e-1279-4095-a612-db67115479be None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:19.439394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ba8aca6e-1279-4095-a612-db67115479be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:19.457626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd3d03-4a4c-4b42-9399-7956b33bf143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:19.458420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd3d03-4a4c-4b42-9399-7956b33bf143 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.458704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd3d03-4a4c-4b42-9399-7956b33bf143 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.458979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd3d03-4a4c-4b42-9399-7956b33bf143 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.465609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd3d03-4a4c-4b42-9399-7956b33bf143 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:19.467066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd3d03-4a4c-4b42-9399-7956b33bf143 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-aacd3d03-4a4c-4b42-9399-7956b33bf143 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:19.481056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:19.481988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:19.482883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:19.482883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:19.482883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:19.482883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:19.482883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:19.482883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:19.482883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:19.482883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:19.482883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 327/638] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:19.483359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd3d03-4a4c-4b42-9399-7956b33bf143 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:19.494277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:19.509546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 312/639] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:19.510354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8aca6e-1279-4095-a612-db67115479be None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:19.541197 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:19.590945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd3d03-4a4c-4b42-9399-7956b33bf143 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:19.609884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8aca6e-1279-4095-a612-db67115479be None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:19.623768 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909921a-d588-41b1-bfb9-6e1a366db966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:19.624593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909921a-d588-41b1-bfb9-6e1a366db966 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.624884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909921a-d588-41b1-bfb9-6e1a366db966 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.625163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909921a-d588-41b1-bfb9-6e1a366db966 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.634966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909921a-d588-41b1-bfb9-6e1a366db966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:19.636988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909921a-d588-41b1-bfb9-6e1a366db966 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:19.644792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d15de-f5b4-4b13-9df3-eb166b307630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:19.645671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d15de-f5b4-4b13-9df3-eb166b307630 None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.645949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d15de-f5b4-4b13-9df3-eb166b307630 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.646218 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d15de-f5b4-4b13-9df3-eb166b307630 None admin] PATH_INFO: `/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/004e5909f9834cfea890063bb0083d7c/roles/2a15b4ff19204ac5916a0099ec8660c6` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-f909921a-d588-41b1-bfb9-6e1a366db966 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:19.654917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:19.655816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:19.657452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:19.657452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:19.657452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:19.657452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:19.657452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 328/640] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:19.657452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909921a-d588-41b1-bfb9-6e1a366db966 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:19.665497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d15de-f5b4-4b13-9df3-eb166b307630 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4ec9b09302d4c918ef638d794468ba3, user_id=004e5909f9834cfea890063bb0083d7c, role_id=2a15b4ff19204ac5916a0099ec8660c6)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:19.666992 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d15de-f5b4-4b13-9df3-eb166b307630 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:19.680056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 313/641] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:07:19 2022] PUT /identity/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/004e5909f9834cfea890063bb0083d7c/roles/2a15b4ff19204ac5916a0099ec8660c6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:07:19.680867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d15de-f5b4-4b13-9df3-eb166b307630 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:19.735245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:19.779245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909921a-d588-41b1-bfb9-6e1a366db966 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:19.785054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:19.810783 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d15de-f5b4-4b13-9df3-eb166b307630 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:19.816336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac8b4e-52db-4c21-89d8-147dc49e1d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:19.817153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac8b4e-52db-4c21-89d8-147dc49e1d90 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.817437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac8b4e-52db-4c21-89d8-147dc49e1d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.817723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac8b4e-52db-4c21-89d8-147dc49e1d90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.830952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac8b4e-52db-4c21-89d8-147dc49e1d90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-92ac8b4e-52db-4c21-89d8-147dc49e1d90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:19.840096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:19.841135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 329/642] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:19.842397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac8b4e-52db-4c21-89d8-147dc49e1d90 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:19.851466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db671e-9a48-4f26-b32b-9cc918f1f822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:19.853027 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db671e-9a48-4f26-b32b-9cc918f1f822 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.853529 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db671e-9a48-4f26-b32b-9cc918f1f822 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.854551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db671e-9a48-4f26-b32b-9cc918f1f822 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:19.856055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db671e-9a48-4f26-b32b-9cc918f1f822 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:19.856772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db671e-9a48-4f26-b32b-9cc918f1f822 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:19.858895 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db671e-9a48-4f26-b32b-9cc918f1f822 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:19.871853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 314/643] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:19.873088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db671e-9a48-4f26-b32b-9cc918f1f822 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:19.952031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac8b4e-52db-4c21-89d8-147dc49e1d90 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:19.975284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db671e-9a48-4f26-b32b-9cc918f1f822 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:19.977154 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:19.991086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e899a9-662b-4ce8-8beb-e4c27f46b5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:19.991897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e899a9-662b-4ce8-8beb-e4c27f46b5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:19.992219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e899a9-662b-4ce8-8beb-e4c27f46b5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:19.992500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e899a9-662b-4ce8-8beb-e4c27f46b5d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:20.002844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e899a9-662b-4ce8-8beb-e4c27f46b5d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:20.010224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce3645-02e9-48ca-aade-08a7265bf621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-b0e899a9-662b-4ce8-8beb-e4c27f46b5d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:20.010689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:20.012203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:20.013660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:20.013660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:20.013660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:20.013660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:20.013660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:20.013660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:20.013660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:20.013660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:20.013660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:20.013660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:20.013660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:20.013660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:20.014420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce3645-02e9-48ca-aade-08a7265bf621 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:20.014420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce3645-02e9-48ca-aade-08a7265bf621 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:20.014420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce3645-02e9-48ca-aade-08a7265bf621 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:20.014615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 330/644] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:20.014615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e899a9-662b-4ce8-8beb-e4c27f46b5d3 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:20.016414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce3645-02e9-48ca-aade-08a7265bf621 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:20.017063 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce3645-02e9-48ca-aade-08a7265bf621 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:20.019810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce3645-02e9-48ca-aade-08a7265bf621 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:20.038915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 315/645] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:20.038915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce3645-02e9-48ca-aade-08a7265bf621 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:20.042378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:20.119916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e899a9-662b-4ce8-8beb-e4c27f46b5d3 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:20.143474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce3645-02e9-48ca-aade-08a7265bf621 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:20.164344 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ee8b0-25c8-49a1-961c-2093f3613249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:20.165025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ee8b0-25c8-49a1-961c-2093f3613249 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:20.165315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ee8b0-25c8-49a1-961c-2093f3613249 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:20.165600 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ee8b0-25c8-49a1-961c-2093f3613249 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:20.166781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ee8b0-25c8-49a1-961c-2093f3613249 None admin] RBAC: Adding query filter params (name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:20.167300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ee8b0-25c8-49a1-961c-2093f3613249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:20.168705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ee8b0-25c8-49a1-961c-2093f3613249 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:20.176982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 331/646] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:20.177601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ee8b0-25c8-49a1-961c-2093f3613249 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:20.185621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f671c-fc0e-4b85-bdbe-2d2ab2da752f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:20.186286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f671c-fc0e-4b85-bdbe-2d2ab2da752f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:20.186572 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f671c-fc0e-4b85-bdbe-2d2ab2da752f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:20.187167 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f671c-fc0e-4b85-bdbe-2d2ab2da752f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:20.187685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f671c-fc0e-4b85-bdbe-2d2ab2da752f None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=8d3baa16824f40f3a4c783e210b262f4) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:20.188002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f671c-fc0e-4b85-bdbe-2d2ab2da752f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:20.189474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f671c-fc0e-4b85-bdbe-2d2ab2da752f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:20.204174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 316/647] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:20 2022] GET /identity/v3/role_assignments?user.id=8d3baa16824f40f3a4c783e210b262f4&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:20.204174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f671c-fc0e-4b85-bdbe-2d2ab2da752f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:20.238322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:20.282870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:20.313348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ee8b0-25c8-49a1-961c-2093f3613249 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:20.320433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f671c-fc0e-4b85-bdbe-2d2ab2da752f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:20.357813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053f3b5-aa0d-4644-86a0-678402edeb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:20.358718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053f3b5-aa0d-4644-86a0-678402edeb72 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:20.359007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053f3b5-aa0d-4644-86a0-678402edeb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:20.359338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053f3b5-aa0d-4644-86a0-678402edeb72 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:20.360789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebee95-3560-437a-9f97-1342f35d90ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:20.361797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebee95-3560-437a-9f97-1342f35d90ef None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:20.362743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebee95-3560-437a-9f97-1342f35d90ef None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:20.363153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebee95-3560-437a-9f97-1342f35d90ef None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:20.364023 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebee95-3560-437a-9f97-1342f35d90ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:20.364458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebee95-3560-437a-9f97-1342f35d90ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:20.365965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebee95-3560-437a-9f97-1342f35d90ef None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:20.368129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053f3b5-aa0d-4644-86a0-678402edeb72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:20.369736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053f3b5-aa0d-4644-86a0-678402edeb72 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:20.377812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 332/648] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:20.378921 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebee95-3560-437a-9f97-1342f35d90ef None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-2053f3b5-aa0d-4644-86a0-678402edeb72 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:20.386807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:20.388007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:20.389461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:20.389461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:20.389461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:20.389461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:20.389461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:20.389461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:20.389461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:20.389461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:20.389461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 17 07:07:20.389461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:20.389461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 317/649] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:20.389461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053f3b5-aa0d-4644-86a0-678402edeb72 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:20.508661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053f3b5-aa0d-4644-86a0-678402edeb72 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:20.521114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebee95-3560-437a-9f97-1342f35d90ef None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:20.530052 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:20.544100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:20.556698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5dffa-71da-4f31-afe4-956deb544e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:20.557473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5dffa-71da-4f31-afe4-956deb544e93 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:20.557930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5dffa-71da-4f31-afe4-956deb544e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:20.558277 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5dffa-71da-4f31-afe4-956deb544e93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:20.566347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de43794-c251-4e77-8903-d1a68429575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:20.567122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de43794-c251-4e77-8903-d1a68429575a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:20.567408 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de43794-c251-4e77-8903-d1a68429575a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:20.567685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de43794-c251-4e77-8903-d1a68429575a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:20.574954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5dffa-71da-4f31-afe4-956deb544e93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:20.576464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5dffa-71da-4f31-afe4-956deb544e93 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:20.583055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de43794-c251-4e77-8903-d1a68429575a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:20.584705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de43794-c251-4e77-8903-d1a68429575a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-fea5dffa-71da-4f31-afe4-956deb544e93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:20.594871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:20.595943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:20.597063 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:20.597063 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:20.597063 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:20.597063 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:20.597063 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 318/650] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:20.597394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5dffa-71da-4f31-afe4-956deb544e93 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-7de43794-c251-4e77-8903-d1a68429575a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:20.606589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:20.607676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:20.608840 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:20.608840 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:20.608840 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:20.608840 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:20.610756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 333/651] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:20.611682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de43794-c251-4e77-8903-d1a68429575a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:20.722695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5dffa-71da-4f31-afe4-956deb544e93 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:20.729641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de43794-c251-4e77-8903-d1a68429575a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:20.767693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea133c95-9488-417a-bbfb-e53e20e54ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:20.767903 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134e4ec-0fe8-4d14-a1e3-a61d27bc5890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:20.768339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134e4ec-0fe8-4d14-a1e3-a61d27bc5890 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:20.768626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134e4ec-0fe8-4d14-a1e3-a61d27bc5890 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:20.768904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134e4ec-0fe8-4d14-a1e3-a61d27bc5890 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:20.769226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea133c95-9488-417a-bbfb-e53e20e54ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:20.769665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea133c95-9488-417a-bbfb-e53e20e54ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:20.769791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134e4ec-0fe8-4d14-a1e3-a61d27bc5890 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:20.770041 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134e4ec-0fe8-4d14-a1e3-a61d27bc5890 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:20.770354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea133c95-9488-417a-bbfb-e53e20e54ff6 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:20.771228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea133c95-9488-417a-bbfb-e53e20e54ff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:20.771491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134e4ec-0fe8-4d14-a1e3-a61d27bc5890 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:20.771841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea133c95-9488-417a-bbfb-e53e20e54ff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:20.773688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea133c95-9488-417a-bbfb-e53e20e54ff6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:20.792025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 334/652] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:20.792683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 319/653] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jun 17 07:07:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:20.793532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea133c95-9488-417a-bbfb-e53e20e54ff6 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:20.796994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134e4ec-0fe8-4d14-a1e3-a61d27bc5890 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:20.819514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:20.835142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:20.916978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134e4ec-0fe8-4d14-a1e3-a61d27bc5890 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:20.920021 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea133c95-9488-417a-bbfb-e53e20e54ff6 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:20.954855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096ce36-8721-4bfa-834f-019ce6365707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:20.956323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096ce36-8721-4bfa-834f-019ce6365707 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:20.956934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096ce36-8721-4bfa-834f-019ce6365707 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:20.957358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096ce36-8721-4bfa-834f-019ce6365707 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:20.958326 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096ce36-8721-4bfa-834f-019ce6365707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:20.958856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096ce36-8721-4bfa-834f-019ce6365707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:20.959715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2104e-fecd-476e-b982-8611ff8f315d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:20.960616 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2104e-fecd-476e-b982-8611ff8f315d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:20.960754 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096ce36-8721-4bfa-834f-019ce6365707 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:20.961031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2104e-fecd-476e-b982-8611ff8f315d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:20.961378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2104e-fecd-476e-b982-8611ff8f315d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:20.971109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2104e-fecd-476e-b982-8611ff8f315d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:20.972907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2104e-fecd-476e-b982-8611ff8f315d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:20.974871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 335/654] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:20.976330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096ce36-8721-4bfa-834f-019ce6365707 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-35c2104e-fecd-476e-b982-8611ff8f315d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:20.995108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:20.996826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:20.997989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:20.997989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:20.997989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:20.997989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:20.997989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:20.997989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:20.997989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:20.997989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:20.997989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 07:07:20.997989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:21.000306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 320/655] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:21.001341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2104e-fecd-476e-b982-8611ff8f315d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:21.080799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:21.105613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:21.109303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096ce36-8721-4bfa-834f-019ce6365707 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:21.113482 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2104e-fecd-476e-b982-8611ff8f315d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:21.153198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5544b-0e9b-48b1-8c85-1855b36f4ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:21.154663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5544b-0e9b-48b1-8c85-1855b36f4ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:21.155124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5544b-0e9b-48b1-8c85-1855b36f4ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:21.155560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5544b-0e9b-48b1-8c85-1855b36f4ee9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:21.156985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5d0d7-40a6-452b-9e93-7d925d94b480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:21.157733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5d0d7-40a6-452b-9e93-7d925d94b480 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:21.158093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5d0d7-40a6-452b-9e93-7d925d94b480 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:21.158567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5d0d7-40a6-452b-9e93-7d925d94b480 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:21.165897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5544b-0e9b-48b1-8c85-1855b36f4ee9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:21.168160 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5544b-0e9b-48b1-8c85-1855b36f4ee9 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-add5544b-0e9b-48b1-8c85-1855b36f4ee9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:21.186994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:21.188018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:21.189034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:21.189034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:21.189034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:21.189034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:21.189034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:21.189034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:21.189034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:21.189034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:21.189034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 07:07:21.189034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:21.192212 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 321/656] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:21.193207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5544b-0e9b-48b1-8c85-1855b36f4ee9 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:21.244782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5d0d7-40a6-452b-9e93-7d925d94b480 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:21.247054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5d0d7-40a6-452b-9e93-7d925d94b480 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:21.313939 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5544b-0e9b-48b1-8c85-1855b36f4ee9 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:21.345328 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:21.359512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea867dd-71c5-484b-abfc-1a07b7dc222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:21.360586 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea867dd-71c5-484b-abfc-1a07b7dc222f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:21.361009 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea867dd-71c5-484b-abfc-1a07b7dc222f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:21.361436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea867dd-71c5-484b-abfc-1a07b7dc222f None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:21.362446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea867dd-71c5-484b-abfc-1a07b7dc222f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:21.362911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea867dd-71c5-484b-abfc-1a07b7dc222f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:21.364682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea867dd-71c5-484b-abfc-1a07b7dc222f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:21.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 336/657] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:07:21.371933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5d0d7-40a6-452b-9e93-7d925d94b480 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:21.384099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 322/658] 10.222.0.33 () {56 vars in 1139 bytes} [Fri Jun 17 07:07:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:21.385128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea867dd-71c5-484b-abfc-1a07b7dc222f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:21.403241 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:21.485148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5d0d7-40a6-452b-9e93-7d925d94b480 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:21.497615 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea867dd-71c5-484b-abfc-1a07b7dc222f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:21.522734 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbbdc1-145e-4210-a989-2efc89f03550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:21.523605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbbdc1-145e-4210-a989-2efc89f03550 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:21.523996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbbdc1-145e-4210-a989-2efc89f03550 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:21.524404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbbdc1-145e-4210-a989-2efc89f03550 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:21.525349 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbbdc1-145e-4210-a989-2efc89f03550 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:21.526031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbbdc1-145e-4210-a989-2efc89f03550 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:21.528203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbbdc1-145e-4210-a989-2efc89f03550 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:21.538600 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03205e9c-b919-4ea9-a9b6-625e3bfcc7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:21.539589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03205e9c-b919-4ea9-a9b6-625e3bfcc7bb None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:21.540041 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03205e9c-b919-4ea9-a9b6-625e3bfcc7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:21.540489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03205e9c-b919-4ea9-a9b6-625e3bfcc7bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:21.553594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 337/659] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jun 17 07:07:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:21.554659 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbbdc1-145e-4210-a989-2efc89f03550 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:21.556046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03205e9c-b919-4ea9-a9b6-625e3bfcc7bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:21.557903 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03205e9c-b919-4ea9-a9b6-625e3bfcc7bb None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-03205e9c-b919-4ea9-a9b6-625e3bfcc7bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:21.578358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:21.579358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:21.580281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:21.580281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:21.580281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:21.580281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:21.582648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 323/660] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:21.583696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03205e9c-b919-4ea9-a9b6-625e3bfcc7bb None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:21.595134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:21.668041 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:21.684480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbbdc1-145e-4210-a989-2efc89f03550 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:21.705056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03205e9c-b919-4ea9-a9b6-625e3bfcc7bb None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:21.730847 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7ed8f-8c87-4ca0-ba79-4bdac230f74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:21.731627 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7ed8f-8c87-4ca0-ba79-4bdac230f74c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:21.731978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7ed8f-8c87-4ca0-ba79-4bdac230f74c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:21.732346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7ed8f-8c87-4ca0-ba79-4bdac230f74c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:21.747434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d856fa-a4d7-4df1-854c-c4b938e58e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:21.748788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d856fa-a4d7-4df1-854c-c4b938e58e7d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:21.749262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d856fa-a4d7-4df1-854c-c4b938e58e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:21.749938 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d856fa-a4d7-4df1-854c-c4b938e58e7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:21.816800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7ed8f-8c87-4ca0-ba79-4bdac230f74c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:21.819494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7ed8f-8c87-4ca0-ba79-4bdac230f74c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:21.835384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d856fa-a4d7-4df1-854c-c4b938e58e7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:21.837153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d856fa-a4d7-4df1-854c-c4b938e58e7d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:21.858008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:21.907963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:21.927366 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 338/661] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:07:21.928527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7ed8f-8c87-4ca0-ba79-4bdac230f74c None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:21.959963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 324/662] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:07:21.962077 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec1766-9c75-41b7-83e6-04bd8f9b0ca0 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:21.962372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec1766-9c75-41b7-83e6-04bd8f9b0ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:21.962651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec1766-9c75-41b7-83e6-04bd8f9b0ca0 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:21.966728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 325/663] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:21.966728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec1766-9c75-41b7-83e6-04bd8f9b0ca0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:22.040967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7ed8f-8c87-4ca0-ba79-4bdac230f74c None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:22.071725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec1766-9c75-41b7-83e6-04bd8f9b0ca0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:22.076266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5f2f4-f6d0-4e81-92c9-e0157ada85a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:22.076936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5f2f4-f6d0-4e81-92c9-e0157ada85a7 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.077226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5f2f4-f6d0-4e81-92c9-e0157ada85a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.077506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5f2f4-f6d0-4e81-92c9-e0157ada85a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.078229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5f2f4-f6d0-4e81-92c9-e0157ada85a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:22.078540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5f2f4-f6d0-4e81-92c9-e0157ada85a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:22.079906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5f2f4-f6d0-4e81-92c9-e0157ada85a7 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:22.082728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:22.094951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 339/664] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:22.094951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5f2f4-f6d0-4e81-92c9-e0157ada85a7 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:22.112781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51754a4a-7c10-4534-b3cf-bd4045a23700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:22.113572 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51754a4a-7c10-4534-b3cf-bd4045a23700 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.113855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51754a4a-7c10-4534-b3cf-bd4045a23700 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.114132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51754a4a-7c10-4534-b3cf-bd4045a23700 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.125205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51754a4a-7c10-4534-b3cf-bd4045a23700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:22.126721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51754a4a-7c10-4534-b3cf-bd4045a23700 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-51754a4a-7c10-4534-b3cf-bd4045a23700 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:22.145993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:22.146930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:22.147832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:22.147832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:22.147832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:22.147832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:22.147832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 326/665] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:22.148585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535aa414-e09b-43ee-9137-99c738993fcc None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.148779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535aa414-e09b-43ee-9137-99c738993fcc None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.149068 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535aa414-e09b-43ee-9137-99c738993fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.166232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-535aa414-e09b-43ee-9137-99c738993fcc None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:22.167427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-535aa414-e09b-43ee-9137-99c738993fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:22.180629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:22.209265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5f2f4-f6d0-4e81-92c9-e0157ada85a7 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:22.237457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 327/666] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:22.238121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535aa414-e09b-43ee-9137-99c738993fcc None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:22.266099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d2d90-dce6-4466-95bb-a6cf097517c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:22.266906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d2d90-dce6-4466-95bb-a6cf097517c0 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.267196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d2d90-dce6-4466-95bb-a6cf097517c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.267482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d2d90-dce6-4466-95bb-a6cf097517c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.279165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d2d90-dce6-4466-95bb-a6cf097517c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:22.280660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d2d90-dce6-4466-95bb-a6cf097517c0 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-968d2d90-dce6-4466-95bb-a6cf097517c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:22.298288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:22.299223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:22.300159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:22.300159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:22.300159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:22.300159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:22.300159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 340/667] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:22.300443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d2d90-dce6-4466-95bb-a6cf097517c0 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:22.351171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535aa414-e09b-43ee-9137-99c738993fcc None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:22.352182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:22.389731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c3268-dc2d-4c01-b13f-5c9d9d889be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:22.390536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c3268-dc2d-4c01-b13f-5c9d9d889be3 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.390823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c3268-dc2d-4c01-b13f-5c9d9d889be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.391101 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c3268-dc2d-4c01-b13f-5c9d9d889be3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.398230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c3268-dc2d-4c01-b13f-5c9d9d889be3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:22.399696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c3268-dc2d-4c01-b13f-5c9d9d889be3 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-141c3268-dc2d-4c01-b13f-5c9d9d889be3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:22.414299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:22.415199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:22.416138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:22.416138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:22.416138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:22.416138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:22.416138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:22.416138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:22.416138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:22.416138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:22.416138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 328/668] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:22.416639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c3268-dc2d-4c01-b13f-5c9d9d889be3 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:22.422804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d2d90-dce6-4466-95bb-a6cf097517c0 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:22.429830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:22.458425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a07f5-ae4f-472f-80a6-7386f2418a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:22.459049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a07f5-ae4f-472f-80a6-7386f2418a3f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.459341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a07f5-ae4f-472f-80a6-7386f2418a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.459624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a07f5-ae4f-472f-80a6-7386f2418a3f None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.460380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a07f5-ae4f-472f-80a6-7386f2418a3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:22.460693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a07f5-ae4f-472f-80a6-7386f2418a3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:22.462078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a07f5-ae4f-472f-80a6-7386f2418a3f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:22.474938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 341/669] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:22.475576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a07f5-ae4f-472f-80a6-7386f2418a3f None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:22.533263 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c3268-dc2d-4c01-b13f-5c9d9d889be3 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:22.568847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aa0ec-4687-4bbc-85d5-5d9ed330eec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:22.569645 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aa0ec-4687-4bbc-85d5-5d9ed330eec2 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.569932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aa0ec-4687-4bbc-85d5-5d9ed330eec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.570205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aa0ec-4687-4bbc-85d5-5d9ed330eec2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.580377 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0aa0ec-4687-4bbc-85d5-5d9ed330eec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:22.584679 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a07f5-ae4f-472f-80a6-7386f2418a3f None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-cc0aa0ec-4687-4bbc-85d5-5d9ed330eec2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:22.588081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:22.589727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 329/670] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:22.590597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aa0ec-4687-4bbc-85d5-5d9ed330eec2 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:22.595764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:22.621263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b0314-718a-457a-8f27-70ccb4d4b83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:22.621924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0314-718a-457a-8f27-70ccb4d4b83f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.622208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0314-718a-457a-8f27-70ccb4d4b83f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.622488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0314-718a-457a-8f27-70ccb4d4b83f None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.623204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b0314-718a-457a-8f27-70ccb4d4b83f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:22.623510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b0314-718a-457a-8f27-70ccb4d4b83f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:22.624919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b0314-718a-457a-8f27-70ccb4d4b83f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:22.635844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 342/671] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:22.636668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b0314-718a-457a-8f27-70ccb4d4b83f None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:22.656293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:22.691189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aa0ec-4687-4bbc-85d5-5d9ed330eec2 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:22.724928 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209ce45-7b7b-4bf6-9aad-334ac5709378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:22.725516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209ce45-7b7b-4bf6-9aad-334ac5709378 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.725807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209ce45-7b7b-4bf6-9aad-334ac5709378 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.726088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209ce45-7b7b-4bf6-9aad-334ac5709378 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.726875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209ce45-7b7b-4bf6-9aad-334ac5709378 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:22.727188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209ce45-7b7b-4bf6-9aad-334ac5709378 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:22.728663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209ce45-7b7b-4bf6-9aad-334ac5709378 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:22.736087 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b0314-718a-457a-8f27-70ccb4d4b83f None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:22.741186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 330/672] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:22.742412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59254e94-de72-4fac-b9a8-f05be4f1a6d3 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.742704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59254e94-de72-4fac-b9a8-f05be4f1a6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.743394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59254e94-de72-4fac-b9a8-f05be4f1a6d3 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.745109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 331/673] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:22.745719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59254e94-de72-4fac-b9a8-f05be4f1a6d3 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:22.770625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f77e7-f3f4-4d93-ac02-6b147856ff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:22.771407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f77e7-f3f4-4d93-ac02-6b147856ff4f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.771690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f77e7-f3f4-4d93-ac02-6b147856ff4f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.771964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f77e7-f3f4-4d93-ac02-6b147856ff4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.779268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f77e7-f3f4-4d93-ac02-6b147856ff4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:22.780771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f77e7-f3f4-4d93-ac02-6b147856ff4f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-eb2f77e7-f3f4-4d93-ac02-6b147856ff4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:22.794675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:22.795605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:22.796963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:22.796963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:22.796963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:22.796963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:22.796963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:22.796963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:22.796963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:22.796963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:22.796963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 343/674] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:22.796963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f77e7-f3f4-4d93-ac02-6b147856ff4f None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:22.823878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:22.850242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59254e94-de72-4fac-b9a8-f05be4f1a6d3 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:22.875719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:22.883478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992d53d-cbe2-4272-acd2-fd1230c66658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:22.884166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992d53d-cbe2-4272-acd2-fd1230c66658 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.884453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992d53d-cbe2-4272-acd2-fd1230c66658 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.884728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992d53d-cbe2-4272-acd2-fd1230c66658 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.885463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992d53d-cbe2-4272-acd2-fd1230c66658 None admin] RBAC: Adding query filter params (name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:22.886286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992d53d-cbe2-4272-acd2-fd1230c66658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:22.887642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992d53d-cbe2-4272-acd2-fd1230c66658 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:22.893529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f77e7-f3f4-4d93-ac02-6b147856ff4f None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:22.895202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 332/675] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:22.895980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992d53d-cbe2-4272-acd2-fd1230c66658 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:22.929490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1c3a1-e807-4117-aee7-571b082f610f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:22.930280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1c3a1-e807-4117-aee7-571b082f610f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.930562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1c3a1-e807-4117-aee7-571b082f610f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.930834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1c3a1-e807-4117-aee7-571b082f610f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.937331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1c3a1-e807-4117-aee7-571b082f610f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:22.938747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1c3a1-e807-4117-aee7-571b082f610f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-dad1c3a1-e807-4117-aee7-571b082f610f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:22.952712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:22.953757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:22.954834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:22.954834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:22.954834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:22.954834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:22.954834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:22.954834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:22.954834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:22.954834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:22.954834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 344/676] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:22.955423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cbfea-628c-49c0-b7d5-69969e06b2d9 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:22.955423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cbfea-628c-49c0-b7d5-69969e06b2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:22.955572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cbfea-628c-49c0-b7d5-69969e06b2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:22.971450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-621cbfea-628c-49c0-b7d5-69969e06b2d9 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:22.972640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-621cbfea-628c-49c0-b7d5-69969e06b2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:22.996493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992d53d-cbe2-4272-acd2-fd1230c66658 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:23.029934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721dce0-9514-4763-b604-0ed704cb67f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:23.030577 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721dce0-9514-4763-b604-0ed704cb67f8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:23.030862 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721dce0-9514-4763-b604-0ed704cb67f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:23.031138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721dce0-9514-4763-b604-0ed704cb67f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:23.031951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721dce0-9514-4763-b604-0ed704cb67f8 None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=48586ff1269f49a3ae3d9f3e40c1b9c0) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:23.032286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721dce0-9514-4763-b604-0ed704cb67f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:23.033695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721dce0-9514-4763-b604-0ed704cb67f8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:23.038751 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 345/677] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:23.048148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 333/678] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:22 2022] GET /identity/v3/role_assignments?user.id=48586ff1269f49a3ae3d9f3e40c1b9c0&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:23.048370 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cbfea-628c-49c0-b7d5-69969e06b2d9 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:23.048789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721dce0-9514-4763-b604-0ed704cb67f8 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:23.068817 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:23.117022 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:23.163703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cbfea-628c-49c0-b7d5-69969e06b2d9 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:23.169937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721dce0-9514-4763-b604-0ed704cb67f8 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:23.204078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f365a4c-c7e2-40ec-aefa-a16e9e7c93b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:23.205023 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f365a4c-c7e2-40ec-aefa-a16e9e7c93b0 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:23.205514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f365a4c-c7e2-40ec-aefa-a16e9e7c93b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:23.205796 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f365a4c-c7e2-40ec-aefa-a16e9e7c93b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:23.206591 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f365a4c-c7e2-40ec-aefa-a16e9e7c93b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:23.206905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f365a4c-c7e2-40ec-aefa-a16e9e7c93b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:23.208956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f365a4c-c7e2-40ec-aefa-a16e9e7c93b0 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:23.209082 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e37dbe-b434-4757-9aa3-9287c96bae78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:23.209460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e37dbe-b434-4757-9aa3-9287c96bae78 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:23.209748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e37dbe-b434-4757-9aa3-9287c96bae78 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:23.210046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e37dbe-b434-4757-9aa3-9287c96bae78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:23.222091 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 346/679] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:23.222936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f365a4c-c7e2-40ec-aefa-a16e9e7c93b0 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:23.223356 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e37dbe-b434-4757-9aa3-9287c96bae78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:23.224868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e37dbe-b434-4757-9aa3-9287c96bae78 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-b5e37dbe-b434-4757-9aa3-9287c96bae78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:23.244531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:23.246186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:23.247553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:23.247553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:23.247553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:23.247553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:23.247553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 334/680] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:23.247553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e37dbe-b434-4757-9aa3-9287c96bae78 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:23.313123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:23.337409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f365a4c-c7e2-40ec-aefa-a16e9e7c93b0 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:23.355635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e37dbe-b434-4757-9aa3-9287c96bae78 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:23.364891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:23.375724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058845c-5ee3-447d-8484-ee7f08247a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:23.376415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058845c-5ee3-447d-8484-ee7f08247a92 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:23.376703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058845c-5ee3-447d-8484-ee7f08247a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:23.376982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058845c-5ee3-447d-8484-ee7f08247a92 None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:23.377771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058845c-5ee3-447d-8484-ee7f08247a92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:23.378515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058845c-5ee3-447d-8484-ee7f08247a92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:23.379951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058845c-5ee3-447d-8484-ee7f08247a92 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:23.397610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 347/681] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:23.398752 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058845c-5ee3-447d-8484-ee7f08247a92 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:23.399327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a210b-614f-4355-b1c8-5fb2b4260280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:23.400810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a210b-614f-4355-b1c8-5fb2b4260280 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:23.401105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a210b-614f-4355-b1c8-5fb2b4260280 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:23.401381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a210b-614f-4355-b1c8-5fb2b4260280 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:23.411724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a210b-614f-4355-b1c8-5fb2b4260280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-4f4a210b-614f-4355-b1c8-5fb2b4260280 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:23.419451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:23.420385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:23.421308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 335/682] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:07:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:23.421994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a210b-614f-4355-b1c8-5fb2b4260280 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:23.515378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058845c-5ee3-447d-8484-ee7f08247a92 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:23.530817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a210b-614f-4355-b1c8-5fb2b4260280 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:23.551472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3be90d-6b56-415a-8072-ca0a8b779bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:23.552162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3be90d-6b56-415a-8072-ca0a8b779bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:23.552454 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3be90d-6b56-415a-8072-ca0a8b779bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:23.552735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3be90d-6b56-415a-8072-ca0a8b779bd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:23.553571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3be90d-6b56-415a-8072-ca0a8b779bd4 None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=7e0797679c07454abc867cd7b30c8b56) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:23.553886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3be90d-6b56-415a-8072-ca0a8b779bd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:23.555315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3be90d-6b56-415a-8072-ca0a8b779bd4 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:23.563080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 348/683] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:23 2022] GET /identity/v3/role_assignments?user.id=7e0797679c07454abc867cd7b30c8b56&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:23.563972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3be90d-6b56-415a-8072-ca0a8b779bd4 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:23.573374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:23.583301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c975193-0d9c-4498-860d-8832efd6063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:23.583994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c975193-0d9c-4498-860d-8832efd6063c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:23.584337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c975193-0d9c-4498-860d-8832efd6063c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:23.584621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c975193-0d9c-4498-860d-8832efd6063c None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:23.585375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c975193-0d9c-4498-860d-8832efd6063c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:23.585688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c975193-0d9c-4498-860d-8832efd6063c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:23.587122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c975193-0d9c-4498-860d-8832efd6063c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:23.600625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 336/684] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:23.601340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c975193-0d9c-4498-860d-8832efd6063c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:23.633870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:23.716873 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3be90d-6b56-415a-8072-ca0a8b779bd4 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:23.719394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c975193-0d9c-4498-860d-8832efd6063c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:23.765853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04dc44-293c-470a-a23b-8b0b3a1516a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:23.766464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04dc44-293c-470a-a23b-8b0b3a1516a5 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:23.766760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04dc44-293c-470a-a23b-8b0b3a1516a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:23.767131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04dc44-293c-470a-a23b-8b0b3a1516a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:23.767994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04dc44-293c-470a-a23b-8b0b3a1516a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:23.768343 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04dc44-293c-470a-a23b-8b0b3a1516a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:23.768594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004351c-6932-4fc7-a004-95f8142c304b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:23.769257 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004351c-6932-4fc7-a004-95f8142c304b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:23.769546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004351c-6932-4fc7-a004-95f8142c304b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:23.769712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04dc44-293c-470a-a23b-8b0b3a1516a5 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:23.769864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004351c-6932-4fc7-a004-95f8142c304b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:23.770659 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004351c-6932-4fc7-a004-95f8142c304b None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=3769534c610b448db693f7c5d6133e65) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:23.770982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004351c-6932-4fc7-a004-95f8142c304b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:23.772881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004351c-6932-4fc7-a004-95f8142c304b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:23.779945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 349/685] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:23 2022] GET /identity/v3/role_assignments?user.id=3769534c610b448db693f7c5d6133e65&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:23.780346 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 337/686] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:23.784124 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004351c-6932-4fc7-a004-95f8142c304b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:23.786110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04dc44-293c-470a-a23b-8b0b3a1516a5 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:23.882631 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:23.894863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:23.911050 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04dc44-293c-470a-a23b-8b0b3a1516a5 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:23.922051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004351c-6932-4fc7-a004-95f8142c304b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:23.949334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f59b60-20d2-48bd-816d-6bf276e87ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:23.950255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f59b60-20d2-48bd-816d-6bf276e87ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:23.950549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f59b60-20d2-48bd-816d-6bf276e87ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:23.950833 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f59b60-20d2-48bd-816d-6bf276e87ac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:23.958788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363be9a-5bdb-4de9-990e-ff0ca1d3703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:23.959819 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363be9a-5bdb-4de9-990e-ff0ca1d3703d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:23.960129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363be9a-5bdb-4de9-990e-ff0ca1d3703d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:23.960468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363be9a-5bdb-4de9-990e-ff0ca1d3703d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:23.964317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f59b60-20d2-48bd-816d-6bf276e87ac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:23.965801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f59b60-20d2-48bd-816d-6bf276e87ac8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:23.969315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363be9a-5bdb-4de9-990e-ff0ca1d3703d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:23.970761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363be9a-5bdb-4de9-990e-ff0ca1d3703d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-c363be9a-5bdb-4de9-990e-ff0ca1d3703d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:23.987506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:23.988571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:23.989514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:23.989514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:23.989514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:23.989514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:23.989514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:23.989514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:23.989514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:23.989514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:23.989514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 07:07:23.989514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:23.989514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 350/687] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:07:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-c7f59b60-20d2-48bd-816d-6bf276e87ac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:23.994153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:23.996857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:23.998973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:23.998973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:23.998973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:23.998973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:23.998973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 338/688] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:24.001180 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363be9a-5bdb-4de9-990e-ff0ca1d3703d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:24.002681 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f59b60-20d2-48bd-816d-6bf276e87ac8 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:24.124616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363be9a-5bdb-4de9-990e-ff0ca1d3703d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:24.146353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f59b60-20d2-48bd-816d-6bf276e87ac8 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:24.155736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:24.168923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941aaf9-ee8c-4dd0-ac78-f4fe83565417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:24.169695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941aaf9-ee8c-4dd0-ac78-f4fe83565417 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:24.170051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941aaf9-ee8c-4dd0-ac78-f4fe83565417 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:24.170417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941aaf9-ee8c-4dd0-ac78-f4fe83565417 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:24.171327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941aaf9-ee8c-4dd0-ac78-f4fe83565417 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:24.171710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941aaf9-ee8c-4dd0-ac78-f4fe83565417 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:24.173271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941aaf9-ee8c-4dd0-ac78-f4fe83565417 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:24.192132 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 351/689] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jun 17 07:07:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:24.197481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941aaf9-ee8c-4dd0-ac78-f4fe83565417 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:24.201820 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c272b82-ea91-407b-bcfb-06d171759a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:24.202490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c272b82-ea91-407b-bcfb-06d171759a6b None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:24.202772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c272b82-ea91-407b-bcfb-06d171759a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:24.203049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c272b82-ea91-407b-bcfb-06d171759a6b None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:24.204459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c272b82-ea91-407b-bcfb-06d171759a6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:24.204783 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c272b82-ea91-407b-bcfb-06d171759a6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:24.206216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c272b82-ea91-407b-bcfb-06d171759a6b None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:24.217980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:24.224385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 339/690] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:24.229498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c272b82-ea91-407b-bcfb-06d171759a6b None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:24.339626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941aaf9-ee8c-4dd0-ac78-f4fe83565417 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:24.362465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c272b82-ea91-407b-bcfb-06d171759a6b None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:24.382700 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122ffa2-afbf-4e58-919b-a9921dd39175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:24.383292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122ffa2-afbf-4e58-919b-a9921dd39175 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:24.383589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122ffa2-afbf-4e58-919b-a9921dd39175 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:24.383870 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122ffa2-afbf-4e58-919b-a9921dd39175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:24.416192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1804d-a562-497c-9e2f-53d364da0cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:24.417580 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1804d-a562-497c-9e2f-53d364da0cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:24.417862 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1804d-a562-497c-9e2f-53d364da0cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:24.418135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1804d-a562-497c-9e2f-53d364da0cb6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:24.434991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1804d-a562-497c-9e2f-53d364da0cb6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:24.438194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1804d-a562-497c-9e2f-53d364da0cb6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-f5f1804d-a562-497c-9e2f-53d364da0cb6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:24.468195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:24.469207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:24.470232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:24.470232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:24.470232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:24.470232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:24.470232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:24.470232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:24.470232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:24.470232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:24.470232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:07:24.470232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:24.475733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 340/691] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jun 17 07:07:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:24.480650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:24.481853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1804d-a562-497c-9e2f-53d364da0cb6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:24.516218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122ffa2-afbf-4e58-919b-a9921dd39175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:24.517668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122ffa2-afbf-4e58-919b-a9921dd39175 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:24.588247 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:24.633524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 352/692] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 07:07:24.638042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1804d-a562-497c-9e2f-53d364da0cb6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:24.641021 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122ffa2-afbf-4e58-919b-a9921dd39175 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:24.700492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ffd459-5be3-4d69-b8eb-3b7252f544e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:24.701178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffd459-5be3-4d69-b8eb-3b7252f544e4 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:24.701465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffd459-5be3-4d69-b8eb-3b7252f544e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:24.701741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffd459-5be3-4d69-b8eb-3b7252f544e4 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:24.702592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffd459-5be3-4d69-b8eb-3b7252f544e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:24.702905 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffd459-5be3-4d69-b8eb-3b7252f544e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:24.704377 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffd459-5be3-4d69-b8eb-3b7252f544e4 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:24.743734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 341/693] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jun 17 07:07:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:24.749380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ffd459-5be3-4d69-b8eb-3b7252f544e4 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:24.764097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:24.780722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122ffa2-afbf-4e58-919b-a9921dd39175 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:24.818387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1b554-9f97-458c-8eaa-e1cb36e24a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:24.819242 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1b554-9f97-458c-8eaa-e1cb36e24a40 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:24.819529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1b554-9f97-458c-8eaa-e1cb36e24a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:24.819820 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1b554-9f97-458c-8eaa-e1cb36e24a40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:24.842717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1b554-9f97-458c-8eaa-e1cb36e24a40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:24.844187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1b554-9f97-458c-8eaa-e1cb36e24a40 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-6de1b554-9f97-458c-8eaa-e1cb36e24a40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:24.877341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:24.878277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:24.879172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:24.879172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:24.879172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:24.879172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:24.879172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 353/694] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:24.888280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1b554-9f97-458c-8eaa-e1cb36e24a40 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:24.914109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ffd459-5be3-4d69-b8eb-3b7252f544e4 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:24.934419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:24.956325 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f79719-ef17-45e0-aa82-43a8b3b8efe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:24.957339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f79719-ef17-45e0-aa82-43a8b3b8efe4 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:24.957865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f79719-ef17-45e0-aa82-43a8b3b8efe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:24.958404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f79719-ef17-45e0-aa82-43a8b3b8efe4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:25.055054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f79719-ef17-45e0-aa82-43a8b3b8efe4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:25.056555 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f79719-ef17-45e0-aa82-43a8b3b8efe4 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:25.061104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1b554-9f97-458c-8eaa-e1cb36e24a40 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:25.129748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867666a-751c-475d-8849-fa54ea40dc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:25.130550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867666a-751c-475d-8849-fa54ea40dc17 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:25.130724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867666a-751c-475d-8849-fa54ea40dc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:25.130975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867666a-751c-475d-8849-fa54ea40dc17 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:25.131704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867666a-751c-475d-8849-fa54ea40dc17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:25.132015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867666a-751c-475d-8849-fa54ea40dc17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:25.133430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867666a-751c-475d-8849-fa54ea40dc17 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:25.150105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:25.159178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 354/695] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:25.166134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867666a-751c-475d-8849-fa54ea40dc17 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:25.271204 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 342/696] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 07:07:25.281037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f79719-ef17-45e0-aa82-43a8b3b8efe4 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:25.292798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867666a-751c-475d-8849-fa54ea40dc17 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:25.327244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172e818-aa16-4803-9b61-f5281299ce27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:25.328043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172e818-aa16-4803-9b61-f5281299ce27 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:25.328358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172e818-aa16-4803-9b61-f5281299ce27 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:25.328637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172e818-aa16-4803-9b61-f5281299ce27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:25.335389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172e818-aa16-4803-9b61-f5281299ce27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:25.336869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172e818-aa16-4803-9b61-f5281299ce27 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:25.341098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-5172e818-aa16-4803-9b61-f5281299ce27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:25.351274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:25.352208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:25.353131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:25.353131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:25.353131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:25.353131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:25.353131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:25.353131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:25.353131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:25.353131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:25.353131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 355/697] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:25.358305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172e818-aa16-4803-9b61-f5281299ce27 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:25.440481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:25.476841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172e818-aa16-4803-9b61-f5281299ce27 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:25.513016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0c5e7-eeaf-4f1d-830a-7a3a27ce94ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:25.513684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0c5e7-eeaf-4f1d-830a-7a3a27ce94ce None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:25.513972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0c5e7-eeaf-4f1d-830a-7a3a27ce94ce None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:25.514253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0c5e7-eeaf-4f1d-830a-7a3a27ce94ce None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:25.515048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0c5e7-eeaf-4f1d-830a-7a3a27ce94ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:25.515357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0c5e7-eeaf-4f1d-830a-7a3a27ce94ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:25.516810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0c5e7-eeaf-4f1d-830a-7a3a27ce94ce None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:25.523419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f79719-ef17-45e0-aa82-43a8b3b8efe4 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:25.529965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 356/698] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:25.531362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971bca8-dbf9-4097-a89f-adf236c6a94f None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:25.536401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971bca8-dbf9-4097-a89f-adf236c6a94f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:25.536733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971bca8-dbf9-4097-a89f-adf236c6a94f None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:25.538102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 357/699] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:25.544087 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971bca8-dbf9-4097-a89f-adf236c6a94f None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:25.573935 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889cd37-20c4-42c4-a7a5-d35ee957ec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:25.574975 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889cd37-20c4-42c4-a7a5-d35ee957ec36 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:25.575259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889cd37-20c4-42c4-a7a5-d35ee957ec36 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:25.575943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889cd37-20c4-42c4-a7a5-d35ee957ec36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:25.588941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889cd37-20c4-42c4-a7a5-d35ee957ec36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:25.590407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889cd37-20c4-42c4-a7a5-d35ee957ec36 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-b889cd37-20c4-42c4-a7a5-d35ee957ec36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:25.612729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:25.613672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:25.614562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:25.614562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:25.614562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:25.614562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:25.614562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 343/700] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:25.618105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e4662-cb33-4a95-83b2-c2145f53e9d4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:25.618105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e4662-cb33-4a95-83b2-c2145f53e9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:25.618105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e4662-cb33-4a95-83b2-c2145f53e9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:25.633781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-f67e4662-cb33-4a95-83b2-c2145f53e9d4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:25.634968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f67e4662-cb33-4a95-83b2-c2145f53e9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:25.673148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971bca8-dbf9-4097-a89f-adf236c6a94f None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:25.691246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:25.710759 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be33d11-9873-46b3-8933-185b14b1b7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:25.711271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be33d11-9873-46b3-8933-185b14b1b7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:25.711554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be33d11-9873-46b3-8933-185b14b1b7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:25.711879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be33d11-9873-46b3-8933-185b14b1b7d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:25.712799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be33d11-9873-46b3-8933-185b14b1b7d0 None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=004e5909f9834cfea890063bb0083d7c) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:25.713111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be33d11-9873-46b3-8933-185b14b1b7d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:25.714646 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be33d11-9873-46b3-8933-185b14b1b7d0 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:25.719183 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:25.721404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 358/701] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:25 2022] GET /identity/v3/role_assignments?user.id=004e5909f9834cfea890063bb0083d7c&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:25.722513 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be33d11-9873-46b3-8933-185b14b1b7d0 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:25.727638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 344/702] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:25.779590 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e4662-cb33-4a95-83b2-c2145f53e9d4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:25.926250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be33d11-9873-46b3-8933-185b14b1b7d0 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:25.959949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e4662-cb33-4a95-83b2-c2145f53e9d4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:25.970570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8dba1-4990-4697-8e0f-8fed1f9fae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:25.971253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8dba1-4990-4697-8e0f-8fed1f9fae72 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:25.971588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8dba1-4990-4697-8e0f-8fed1f9fae72 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:25.971878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8dba1-4990-4697-8e0f-8fed1f9fae72 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:25.972635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8dba1-4990-4697-8e0f-8fed1f9fae72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:25.972950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8dba1-4990-4697-8e0f-8fed1f9fae72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:25.974530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8dba1-4990-4697-8e0f-8fed1f9fae72 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:25.989288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 359/703] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:25.994603 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8dba1-4990-4697-8e0f-8fed1f9fae72 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:26.005287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868ea12-4ac0-4056-be4e-dfea74fe6204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:26.006877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868ea12-4ac0-4056-be4e-dfea74fe6204 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:26.007383 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868ea12-4ac0-4056-be4e-dfea74fe6204 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:26.008302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868ea12-4ac0-4056-be4e-dfea74fe6204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:26.021938 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868ea12-4ac0-4056-be4e-dfea74fe6204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:26.024397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868ea12-4ac0-4056-be4e-dfea74fe6204 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:26.041107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-7868ea12-4ac0-4056-be4e-dfea74fe6204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:26.051973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:26.052919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:26.053822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:26.053822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:26.053822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:26.053822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:26.061613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 345/704] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:26.063232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868ea12-4ac0-4056-be4e-dfea74fe6204 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:26.111375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8dba1-4990-4697-8e0f-8fed1f9fae72 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:26.124443 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:26.150300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e09e9-dd42-4732-bb48-c2531a4aa332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:26.151872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e09e9-dd42-4732-bb48-c2531a4aa332 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:26.152525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e09e9-dd42-4732-bb48-c2531a4aa332 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:26.152525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e09e9-dd42-4732-bb48-c2531a4aa332 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:26.160115 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e09e9-dd42-4732-bb48-c2531a4aa332 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:26.161645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e09e9-dd42-4732-bb48-c2531a4aa332 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-c48e09e9-dd42-4732-bb48-c2531a4aa332 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:26.176843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:26.177778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:26.179003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:26.179003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:26.179003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:26.179003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:26.179003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:26.179003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:26.179003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:26.179003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:26.179003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 360/705] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:26.179606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb16e3-ac20-4beb-ab0f-81ef62302816 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:26.179749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb16e3-ac20-4beb-ab0f-81ef62302816 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:26.180032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb16e3-ac20-4beb-ab0f-81ef62302816 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:26.183061 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868ea12-4ac0-4056-be4e-dfea74fe6204 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:26.186722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 361/706] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:26.187697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb16e3-ac20-4beb-ab0f-81ef62302816 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:26.227749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2709a81-6f83-4fdb-9dbb-9c3180860aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:26.228486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2709a81-6f83-4fdb-9dbb-9c3180860aeb None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:26.228772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2709a81-6f83-4fdb-9dbb-9c3180860aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:26.229056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2709a81-6f83-4fdb-9dbb-9c3180860aeb None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:26.230460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2709a81-6f83-4fdb-9dbb-9c3180860aeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:26.230770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2709a81-6f83-4fdb-9dbb-9c3180860aeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:26.232195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2709a81-6f83-4fdb-9dbb-9c3180860aeb None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:26.244618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 346/707] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:26.246867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdf924-49ff-4d60-979f-3ae8ea9d18f0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:26.247162 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdf924-49ff-4d60-979f-3ae8ea9d18f0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:26.247439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdf924-49ff-4d60-979f-3ae8ea9d18f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:26.264157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-9efdf924-49ff-4d60-979f-3ae8ea9d18f0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:26.265335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-9efdf924-49ff-4d60-979f-3ae8ea9d18f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:26.320056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:26.326795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb16e3-ac20-4beb-ab0f-81ef62302816 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:26.336636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 347/708] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:26.337373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdf924-49ff-4d60-979f-3ae8ea9d18f0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:26.372045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47616f4-d1e4-4927-bc1e-c582ec5591eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:26.373343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47616f4-d1e4-4927-bc1e-c582ec5591eb None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:26.374133 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47616f4-d1e4-4927-bc1e-c582ec5591eb None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:26.374662 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47616f4-d1e4-4927-bc1e-c582ec5591eb None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:26.376440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47616f4-d1e4-4927-bc1e-c582ec5591eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:26.377028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47616f4-d1e4-4927-bc1e-c582ec5591eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:26.379685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47616f4-d1e4-4927-bc1e-c582ec5591eb None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:26.398689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:26.407004 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 362/709] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:26.408227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47616f4-d1e4-4927-bc1e-c582ec5591eb None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:26.483803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdf924-49ff-4d60-979f-3ae8ea9d18f0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:26.529639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfb256-7cba-462c-b9fc-cd3f51a41d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:26.530317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfb256-7cba-462c-b9fc-cd3f51a41d44 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:26.530603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfb256-7cba-462c-b9fc-cd3f51a41d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:26.530880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfb256-7cba-462c-b9fc-cd3f51a41d44 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:26.531581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfb256-7cba-462c-b9fc-cd3f51a41d44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:26.533064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfb256-7cba-462c-b9fc-cd3f51a41d44 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:26.540354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47616f4-d1e4-4927-bc1e-c582ec5591eb None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:26.557801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 348/710] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:07:26 2022] POST /identity/v3/users => generated 254 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:26.558816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfb256-7cba-462c-b9fc-cd3f51a41d44 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:26.579340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fddbff-cad0-4d43-9fcf-76f1ddcaac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:26.580191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fddbff-cad0-4d43-9fcf-76f1ddcaac48 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:26.580480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fddbff-cad0-4d43-9fcf-76f1ddcaac48 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:26.580856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fddbff-cad0-4d43-9fcf-76f1ddcaac48 None admin] PATH_INFO: `/v3/services/network` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:26.581529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fddbff-cad0-4d43-9fcf-76f1ddcaac48 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:26.583065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fddbff-cad0-4d43-9fcf-76f1ddcaac48 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-31fddbff-cad0-4d43-9fcf-76f1ddcaac48 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:26.598640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:26.599779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:26.600780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 363/711] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:26 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:26.606892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fddbff-cad0-4d43-9fcf-76f1ddcaac48 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:26.637822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:26.678181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:26.682607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfb256-7cba-462c-b9fc-cd3f51a41d44 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:26.724822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40f691-e48a-4abf-960e-bff157e363bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:26.725503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f691-e48a-4abf-960e-bff157e363bf None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:26.725789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f691-e48a-4abf-960e-bff157e363bf None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:26.726065 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f691-e48a-4abf-960e-bff157e363bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:26.726910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40f691-e48a-4abf-960e-bff157e363bf None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=8d3baa16824f40f3a4c783e210b262f4) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:26.727225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40f691-e48a-4abf-960e-bff157e363bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:26.728743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40f691-e48a-4abf-960e-bff157e363bf None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:26.733230 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fddbff-cad0-4d43-9fcf-76f1ddcaac48 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:26.736132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 349/712] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:26 2022] GET /identity/v3/role_assignments?user.id=8d3baa16824f40f3a4c783e210b262f4&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:26.745798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6bf64-e82a-4db3-952c-e434d95bf56a None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:26.745798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6bf64-e82a-4db3-952c-e434d95bf56a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:26.745798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6bf64-e82a-4db3-952c-e434d95bf56a None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:26.745798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 350/713] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:26.746202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4bd5c-09f0-4870-a8d1-ec70baafaf29 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:26.746464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4bd5c-09f0-4870-a8d1-ec70baafaf29 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:26.750719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4bd5c-09f0-4870-a8d1-ec70baafaf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:26.784918 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-5ee4bd5c-09f0-4870-a8d1-ec70baafaf29 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:26.787057 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5ee4bd5c-09f0-4870-a8d1-ec70baafaf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:26.807202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44a0bc-e8d5-419e-9242-a10d4f3eeb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:26.808941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44a0bc-e8d5-419e-9242-a10d4f3eeb8c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:26.809232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44a0bc-e8d5-419e-9242-a10d4f3eeb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:26.809513 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44a0bc-e8d5-419e-9242-a10d4f3eeb8c None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:26.811381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44a0bc-e8d5-419e-9242-a10d4f3eeb8c None admin] RBAC: Adding query filter params (name=network) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:26.811705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44a0bc-e8d5-419e-9242-a10d4f3eeb8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:26.813167 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44a0bc-e8d5-419e-9242-a10d4f3eeb8c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:26.820847 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 364/714] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:07:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:26.826570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44a0bc-e8d5-419e-9242-a10d4f3eeb8c None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:26.896033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 351/715] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:26.936302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4bd5c-09f0-4870-a8d1-ec70baafaf29 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:26.957839 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:26.975276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44a0bc-e8d5-419e-9242-a10d4f3eeb8c None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:27.015368 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbbf3e-68ef-4b69-b705-e64610d110d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:27.016251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbbf3e-68ef-4b69-b705-e64610d110d7 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:27.016543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbbf3e-68ef-4b69-b705-e64610d110d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:27.017412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbbf3e-68ef-4b69-b705-e64610d110d7 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:27.018129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbbf3e-68ef-4b69-b705-e64610d110d7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:27.018728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbbf3e-68ef-4b69-b705-e64610d110d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:27.020457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbbf3e-68ef-4b69-b705-e64610d110d7 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:27.029198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 365/716] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:07:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:27.048400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:27.059667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4bd5c-09f0-4870-a8d1-ec70baafaf29 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:27.103877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25a993-fe93-4039-98eb-4e1df2b0cb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:27.105302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25a993-fe93-4039-98eb-4e1df2b0cb07 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:27.105589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25a993-fe93-4039-98eb-4e1df2b0cb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:27.105866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25a993-fe93-4039-98eb-4e1df2b0cb07 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:27.106531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25a993-fe93-4039-98eb-4e1df2b0cb07 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:27.107919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25a993-fe93-4039-98eb-4e1df2b0cb07 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-8f25a993-fe93-4039-98eb-4e1df2b0cb07 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:27.129121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:27.130688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:27.131902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 352/717] 10.222.0.33 () {56 vars in 1129 bytes} [Fri Jun 17 07:07:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:27.137250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbbf3e-68ef-4b69-b705-e64610d110d7 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:27.315207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbbf3e-68ef-4b69-b705-e64610d110d7 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:27.350906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da485722-0e3f-4f3d-a8fc-8df7118b7cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:27.351582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da485722-0e3f-4f3d-a8fc-8df7118b7cfd None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:27.351871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da485722-0e3f-4f3d-a8fc-8df7118b7cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:27.352174 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da485722-0e3f-4f3d-a8fc-8df7118b7cfd None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:27.352887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da485722-0e3f-4f3d-a8fc-8df7118b7cfd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:27.353195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da485722-0e3f-4f3d-a8fc-8df7118b7cfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:27.355150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da485722-0e3f-4f3d-a8fc-8df7118b7cfd None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:27.361148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 366/718] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jun 17 07:07:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:27.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25a993-fe93-4039-98eb-4e1df2b0cb07 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:27.539675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25a993-fe93-4039-98eb-4e1df2b0cb07 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:27.590423 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae010607-b315-4c57-9cac-5600f236b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:27.591525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae010607-b315-4c57-9cac-5600f236b25a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:27.592036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae010607-b315-4c57-9cac-5600f236b25a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:27.592590 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae010607-b315-4c57-9cac-5600f236b25a None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:27.593938 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae010607-b315-4c57-9cac-5600f236b25a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:27.594521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae010607-b315-4c57-9cac-5600f236b25a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:27.596386 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae010607-b315-4c57-9cac-5600f236b25a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:27.609068 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 353/719] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jun 17 07:07:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:28.444566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada728e-cc84-40c8-a34e-2ade88dba07e None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:28.445103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada728e-cc84-40c8-a34e-2ade88dba07e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:28.445103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada728e-cc84-40c8-a34e-2ade88dba07e None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:28.446725 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 367/720] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:28.452224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e806b7-00cd-4329-a477-38f9179d6813 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:28.452635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e806b7-00cd-4329-a477-38f9179d6813 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:28.452804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e806b7-00cd-4329-a477-38f9179d6813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:28.474846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-a5e806b7-00cd-4329-a477-38f9179d6813 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:28.476116 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-a5e806b7-00cd-4329-a477-38f9179d6813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:28.490614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:28.562721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 354/721] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:28.597028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada728e-cc84-40c8-a34e-2ade88dba07e None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:28.624093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:28.720238 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada728e-cc84-40c8-a34e-2ade88dba07e None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:28.754772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4a696-2c7b-4620-a48a-644e89a750fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:28.755911 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4a696-2c7b-4620-a48a-644e89a750fc None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:28.756455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4a696-2c7b-4620-a48a-644e89a750fc None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:28.757033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4a696-2c7b-4620-a48a-644e89a750fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:28.768259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4a696-2c7b-4620-a48a-644e89a750fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-1bd4a696-2c7b-4620-a48a-644e89a750fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:28.776554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:28.777500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:28.778348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:28.778348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:28.778348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:28.778348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:28.778348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:28.778348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:28.778348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:28.778348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:28.778348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:28.778348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:28.778348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:07:28.778348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:28.781088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 368/722] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:07:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:28.787013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e806b7-00cd-4329-a477-38f9179d6813 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:28.920567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e806b7-00cd-4329-a477-38f9179d6813 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:28.955934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdc29a-f7f1-462b-bc03-f45508709646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:28.957376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdc29a-f7f1-462b-bc03-f45508709646 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:28.958078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdc29a-f7f1-462b-bc03-f45508709646 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:28.958763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdc29a-f7f1-462b-bc03-f45508709646 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:28.960282 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fdc29a-f7f1-462b-bc03-f45508709646 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:28.960811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fdc29a-f7f1-462b-bc03-f45508709646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:28.962495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fdc29a-f7f1-462b-bc03-f45508709646 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:28.971880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 355/723] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:28.981136 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4a696-2c7b-4620-a48a-644e89a750fc None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:29.079031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:29.110209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4a696-2c7b-4620-a48a-644e89a750fc None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:29.156765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c0f62-959f-42e0-afe1-f2227bd0912e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:29.157865 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c0f62-959f-42e0-afe1-f2227bd0912e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:29.158202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c0f62-959f-42e0-afe1-f2227bd0912e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:29.158496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c0f62-959f-42e0-afe1-f2227bd0912e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:29.171515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c0f62-959f-42e0-afe1-f2227bd0912e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:29.173095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c0f62-959f-42e0-afe1-f2227bd0912e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-8e8c0f62-959f-42e0-afe1-f2227bd0912e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:29.192103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:29.193066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:29.194623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:29.194623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:29.194623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:29.194623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:29.194623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 369/724] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:29.198975 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdc29a-f7f1-462b-bc03-f45508709646 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:29.235026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:29.311786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8999ff-0d59-4d22-a24d-22eb69feccaf None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:29.312137 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8999ff-0d59-4d22-a24d-22eb69feccaf None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:29.312384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8999ff-0d59-4d22-a24d-22eb69feccaf None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:29.314010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 370/725] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:07:29.318749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5ceec-c0f8-4355-9c21-19bfced805e7 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:29.319049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5ceec-c0f8-4355-9c21-19bfced805e7 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:29.319334 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5ceec-c0f8-4355-9c21-19bfced805e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:29.353534 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdc29a-f7f1-462b-bc03-f45508709646 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:29.369151 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-dbb5ceec-c0f8-4355-9c21-19bfced805e7 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:29.370417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-dbb5ceec-c0f8-4355-9c21-19bfced805e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:29.398920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9d932-e251-485e-824e-a55161e143c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:29.400020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9d932-e251-485e-824e-a55161e143c1 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:29.401331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9d932-e251-485e-824e-a55161e143c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:29.401816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9d932-e251-485e-824e-a55161e143c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:29.403401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9d932-e251-485e-824e-a55161e143c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:29.403954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9d932-e251-485e-824e-a55161e143c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:29.405656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9d932-e251-485e-824e-a55161e143c1 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:29.420970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 356/726] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:29.426832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9d932-e251-485e-824e-a55161e143c1 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:29.501928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 371/727] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 07:07:29.503210 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd165c-43b0-44a3-bcb2-e5ecb97b86ed None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:29.503488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd165c-43b0-44a3-bcb2-e5ecb97b86ed None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:29.503769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd165c-43b0-44a3-bcb2-e5ecb97b86ed None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:29.513646 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 372/728] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:29.519558 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777ca70-c1ce-4124-a31f-96536e4d76e0 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:29.519877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777ca70-c1ce-4124-a31f-96536e4d76e0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:29.520302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777ca70-c1ce-4124-a31f-96536e4d76e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:29.524426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:29.550402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3777ca70-c1ce-4124-a31f-96536e4d76e0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:29.551703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3777ca70-c1ce-4124-a31f-96536e4d76e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:29.565446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9d932-e251-485e-824e-a55161e143c1 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:29.602039 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5c3ca-bd84-42a8-a1d7-be4b0b1c9b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:29.602935 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5c3ca-bd84-42a8-a1d7-be4b0b1c9b98 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:29.603342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5c3ca-bd84-42a8-a1d7-be4b0b1c9b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:29.603617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5c3ca-bd84-42a8-a1d7-be4b0b1c9b98 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:29.610356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:29.616675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5c3ca-bd84-42a8-a1d7-be4b0b1c9b98 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:29.621698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5c3ca-bd84-42a8-a1d7-be4b0b1c9b98 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:29.632087 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 373/729] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:29.632087 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777ca70-c1ce-4124-a31f-96536e4d76e0 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-96d5c3ca-bd84-42a8-a1d7-be4b0b1c9b98 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:29.652537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:29.653500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:29.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:29.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:29.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:29.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:29.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:29.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:29.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:29.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:29.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 07:07:29.654460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:29.655015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 357/730] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:07:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:29.661182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5c3ca-bd84-42a8-a1d7-be4b0b1c9b98 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:29.759525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777ca70-c1ce-4124-a31f-96536e4d76e0 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:29.799162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543e355-573e-429f-a9cd-690c5e604343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:29.800085 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543e355-573e-429f-a9cd-690c5e604343 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:29.800399 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543e355-573e-429f-a9cd-690c5e604343 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:29.800682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543e355-573e-429f-a9cd-690c5e604343 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:29.807250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5c3ca-bd84-42a8-a1d7-be4b0b1c9b98 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:29.811663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543e355-573e-429f-a9cd-690c5e604343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-1543e355-573e-429f-a9cd-690c5e604343 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:29.819354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:29.820315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:29.821160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:29.821160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:29.821160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:29.821160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:29.821160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:29.821160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:29.821160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:29.821160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:29.821160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:29.821160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:29.821160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:07:29.821160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:29.824506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 374/731] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:07:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:29.825040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543e355-573e-429f-a9cd-690c5e604343 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:29.840028 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:29.845915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475e03d-a280-4f4d-9184-705ae9a89e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:29.846716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475e03d-a280-4f4d-9184-705ae9a89e9f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:29.847003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475e03d-a280-4f4d-9184-705ae9a89e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:29.847273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475e03d-a280-4f4d-9184-705ae9a89e9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:29.857812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475e03d-a280-4f4d-9184-705ae9a89e9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-f475e03d-a280-4f4d-9184-705ae9a89e9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:29.865502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:29.866458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:29.867305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 358/732] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:29.868583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc52e5-07de-4e07-9732-001bfdf1f626 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:29.868686 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc52e5-07de-4e07-9732-001bfdf1f626 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:29.868883 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc52e5-07de-4e07-9732-001bfdf1f626 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:29.870797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 359/733] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:29.872832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc52e5-07de-4e07-9732-001bfdf1f626 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:29.884902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:29.938874 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543e355-573e-429f-a9cd-690c5e604343 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:29.980415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c649f-1605-4473-87c3-28610fef9563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:29.981196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c649f-1605-4473-87c3-28610fef9563 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:29.981490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c649f-1605-4473-87c3-28610fef9563 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:29.981813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c649f-1605-4473-87c3-28610fef9563 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:29.982651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c649f-1605-4473-87c3-28610fef9563 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:29.982959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c649f-1605-4473-87c3-28610fef9563 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:29.984407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c649f-1605-4473-87c3-28610fef9563 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:29.991074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc52e5-07de-4e07-9732-001bfdf1f626 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:30.004413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 375/734] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jun 17 07:07:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:30.006541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c649f-1605-4473-87c3-28610fef9563 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:30.031375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3886e-c944-4995-8270-ad1a20c794bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:30.031977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3886e-c944-4995-8270-ad1a20c794bf None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:30.032323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3886e-c944-4995-8270-ad1a20c794bf None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:30.032689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3886e-c944-4995-8270-ad1a20c794bf None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:30.033348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3886e-c944-4995-8270-ad1a20c794bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:30.033659 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3886e-c944-4995-8270-ad1a20c794bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:30.035038 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3886e-c944-4995-8270-ad1a20c794bf None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:30.048731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 360/735] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:30.048731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a6cf1-70d9-4a3c-ac35-0678c145862a None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:30.048731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a6cf1-70d9-4a3c-ac35-0678c145862a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:30.048731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a6cf1-70d9-4a3c-ac35-0678c145862a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:30.068088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-9f8a6cf1-70d9-4a3c-ac35-0678c145862a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:30.069270 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-9f8a6cf1-70d9-4a3c-ac35-0678c145862a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:30.115370 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c649f-1605-4473-87c3-28610fef9563 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:30.121452 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:30.124544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:30.140153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 361/736] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:30.140851 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a6cf1-70d9-4a3c-ac35-0678c145862a None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:30.168716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c95d1-3044-4f94-952a-6ec5784a99f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:30.170011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c95d1-3044-4f94-952a-6ec5784a99f7 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:30.170312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c95d1-3044-4f94-952a-6ec5784a99f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:30.170867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c95d1-3044-4f94-952a-6ec5784a99f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:30.171627 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c95d1-3044-4f94-952a-6ec5784a99f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:30.172247 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c95d1-3044-4f94-952a-6ec5784a99f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:30.174368 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c95d1-3044-4f94-952a-6ec5784a99f7 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:30.185004 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 376/737] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:30.189429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c95d1-3044-4f94-952a-6ec5784a99f7 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:30.246385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a6cf1-70d9-4a3c-ac35-0678c145862a None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:30.283854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf39f38-b13f-48b9-99b6-20d3f91cb8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:30.284450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf39f38-b13f-48b9-99b6-20d3f91cb8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:30.284731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf39f38-b13f-48b9-99b6-20d3f91cb8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:30.285011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf39f38-b13f-48b9-99b6-20d3f91cb8d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:30.294845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c95d1-3044-4f94-952a-6ec5784a99f7 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:30.331607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c589d18-65fd-48dc-b648-e545877c3787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:30.332444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c589d18-65fd-48dc-b648-e545877c3787 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:30.332726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c589d18-65fd-48dc-b648-e545877c3787 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:30.333000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c589d18-65fd-48dc-b648-e545877c3787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:30.339642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:30.349146 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c589d18-65fd-48dc-b648-e545877c3787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:30.351560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c589d18-65fd-48dc-b648-e545877c3787 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:30.357533 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf39f38-b13f-48b9-99b6-20d3f91cb8d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:30.359022 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf39f38-b13f-48b9-99b6-20d3f91cb8d5 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-5c589d18-65fd-48dc-b648-e545877c3787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:30.370978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:30.371948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:30.372919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:30.372919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:30.372919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:30.372919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:30.372919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 377/738] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:30.373693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c589d18-65fd-48dc-b648-e545877c3787 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:30.402644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:30.465873 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 362/739] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 07:07:30.466717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf39f38-b13f-48b9-99b6-20d3f91cb8d5 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:30.482637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c589d18-65fd-48dc-b648-e545877c3787 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:30.515074 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0e5e2-c05f-42f7-aaf8-8e952acc1e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:30.515721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0e5e2-c05f-42f7-aaf8-8e952acc1e16 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:30.516003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0e5e2-c05f-42f7-aaf8-8e952acc1e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:30.516303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0e5e2-c05f-42f7-aaf8-8e952acc1e16 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:30.517010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0e5e2-c05f-42f7-aaf8-8e952acc1e16 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:30.518375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0e5e2-c05f-42f7-aaf8-8e952acc1e16 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:30.540349 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 378/740] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:07:30 2022] POST /identity/v3/services => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:30.542038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0e5e2-c05f-42f7-aaf8-8e952acc1e16 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:30.548973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:30.569188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf39f38-b13f-48b9-99b6-20d3f91cb8d5 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:30.617433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29551e9-ab87-452b-9ad5-66459d0a2759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:30.618282 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29551e9-ab87-452b-9ad5-66459d0a2759 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:30.618541 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29551e9-ab87-452b-9ad5-66459d0a2759 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:30.618821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29551e9-ab87-452b-9ad5-66459d0a2759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:30.630302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29551e9-ab87-452b-9ad5-66459d0a2759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:30.631809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29551e9-ab87-452b-9ad5-66459d0a2759 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:30.634096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-a29551e9-ab87-452b-9ad5-66459d0a2759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:30.652692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:30.653683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:30.654602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:30.654602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:30.654602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:30.654602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:30.658807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 363/741] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:30.660231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe9a9f-edf3-4422-af7b-d54477f8f594 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:30.660525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe9a9f-edf3-4422-af7b-d54477f8f594 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:30.660803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe9a9f-edf3-4422-af7b-d54477f8f594 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:30.661589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0e5e2-c05f-42f7-aaf8-8e952acc1e16 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:30.662298 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 364/742] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:30.662864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe9a9f-edf3-4422-af7b-d54477f8f594 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:30.708263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdfdd5-0446-4d57-a55c-a3bca0b38128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:30.708940 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdfdd5-0446-4d57-a55c-a3bca0b38128 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:30.709217 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdfdd5-0446-4d57-a55c-a3bca0b38128 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:30.709496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdfdd5-0446-4d57-a55c-a3bca0b38128 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:30.710224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdfdd5-0446-4d57-a55c-a3bca0b38128 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:30.710533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdfdd5-0446-4d57-a55c-a3bca0b38128 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:30.712159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdfdd5-0446-4d57-a55c-a3bca0b38128 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:30.726107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 379/743] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:30.727328 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdfdd5-0446-4d57-a55c-a3bca0b38128 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:30.778932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe9a9f-edf3-4422-af7b-d54477f8f594 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:30.809800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:30.816359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33994e21-193c-442b-9aeb-8eb9475d111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:30.818194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33994e21-193c-442b-9aeb-8eb9475d111a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:30.818635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33994e21-193c-442b-9aeb-8eb9475d111a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:30.819060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33994e21-193c-442b-9aeb-8eb9475d111a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:30.832731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33994e21-193c-442b-9aeb-8eb9475d111a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:30.834196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33994e21-193c-442b-9aeb-8eb9475d111a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:30.848539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdfdd5-0446-4d57-a55c-a3bca0b38128 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-33994e21-193c-442b-9aeb-8eb9475d111a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:30.852535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:30.853426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:30.854877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:30.854877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:30.854877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:30.854877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:30.858711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 365/744] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:30.859901 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b9995-89ff-43f6-8be6-62ec8f2c1f8b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:30.860220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b9995-89ff-43f6-8be6-62ec8f2c1f8b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:30.860616 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b9995-89ff-43f6-8be6-62ec8f2c1f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:30.877853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-364b9995-89ff-43f6-8be6-62ec8f2c1f8b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:30.879124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-364b9995-89ff-43f6-8be6-62ec8f2c1f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:30.885727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77270877-5fde-4333-9c4c-ce4d5f31fd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:30.886407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77270877-5fde-4333-9c4c-ce4d5f31fd73 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:30.887104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77270877-5fde-4333-9c4c-ce4d5f31fd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:30.887387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77270877-5fde-4333-9c4c-ce4d5f31fd73 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:30.888144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77270877-5fde-4333-9c4c-ce4d5f31fd73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:30.888800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77270877-5fde-4333-9c4c-ce4d5f31fd73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:30.890485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77270877-5fde-4333-9c4c-ce4d5f31fd73 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:30.901854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:30.906662 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 380/745] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:30.907482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77270877-5fde-4333-9c4c-ce4d5f31fd73 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:30.957053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 366/746] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:30 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 07:07:30.957894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b9995-89ff-43f6-8be6-62ec8f2c1f8b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:31.028945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77270877-5fde-4333-9c4c-ce4d5f31fd73 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:31.064590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f317b-eaa6-4212-b86f-fcd676793eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:31.065633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f317b-eaa6-4212-b86f-fcd676793eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:31.065912 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f317b-eaa6-4212-b86f-fcd676793eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:31.066186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f317b-eaa6-4212-b86f-fcd676793eb3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:31.066671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b9995-89ff-43f6-8be6-62ec8f2c1f8b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:31.074073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f317b-eaa6-4212-b86f-fcd676793eb3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:31.075511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f317b-eaa6-4212-b86f-fcd676793eb3 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:31.083800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-4e6f317b-eaa6-4212-b86f-fcd676793eb3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:31.090993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:31.092046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:31.093016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:31.093016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:31.093016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:31.093016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:31.093016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:31.093016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:31.093016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:31.093016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:31.093016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:07:31.093016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:31.093016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 381/747] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jun 17 07:07:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:31.093608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f317b-eaa6-4212-b86f-fcd676793eb3 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:31.108647 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395f189-ec83-4acf-9fa1-d27e0c45da4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:31.109319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395f189-ec83-4acf-9fa1-d27e0c45da4a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:31.109602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395f189-ec83-4acf-9fa1-d27e0c45da4a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:31.109871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395f189-ec83-4acf-9fa1-d27e0c45da4a None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:31.110599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395f189-ec83-4acf-9fa1-d27e0c45da4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:31.110906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395f189-ec83-4acf-9fa1-d27e0c45da4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:31.112307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395f189-ec83-4acf-9fa1-d27e0c45da4a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:31.123894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 367/748] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:31.124691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395f189-ec83-4acf-9fa1-d27e0c45da4a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:31.163767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:31.208190 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f317b-eaa6-4212-b86f-fcd676793eb3 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:31.230088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395f189-ec83-4acf-9fa1-d27e0c45da4a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:31.243496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249312e0-314d-4fdb-8001-f10fdffbad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:31.244440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249312e0-314d-4fdb-8001-f10fdffbad86 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:31.244795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249312e0-314d-4fdb-8001-f10fdffbad86 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:31.245192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249312e0-314d-4fdb-8001-f10fdffbad86 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:31.253471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249312e0-314d-4fdb-8001-f10fdffbad86 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:31.254988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249312e0-314d-4fdb-8001-f10fdffbad86 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:31.266216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf627d-0cb4-45ce-a845-1bc933d6221a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:31.266953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf627d-0cb4-45ce-a845-1bc933d6221a None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:31.267299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf627d-0cb4-45ce-a845-1bc933d6221a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:31.267641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf627d-0cb4-45ce-a845-1bc933d6221a None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:31.268456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf627d-0cb4-45ce-a845-1bc933d6221a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:31.269891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf627d-0cb4-45ce-a845-1bc933d6221a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-249312e0-314d-4fdb-8001-f10fdffbad86 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:31.271129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:31.272206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:31.273373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:31.273373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:31.273373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:31.273373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:31.273373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:31.273373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:31.273373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:31.273373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:31.273373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 07:07:31.273373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:31.279410 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 382/749] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jun 17 07:07:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:31.280232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249312e0-314d-4fdb-8001-f10fdffbad86 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:31.296233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 368/750] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:07:31 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:31.296233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf627d-0cb4-45ce-a845-1bc933d6221a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:31.347881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:31.416859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249312e0-314d-4fdb-8001-f10fdffbad86 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:31.440329 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:31.455497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf627d-0cb4-45ce-a845-1bc933d6221a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:31.461583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5c1c6-2959-4625-8aed-b7d39c5e9d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:31.462262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5c1c6-2959-4625-8aed-b7d39c5e9d38 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:31.462540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5c1c6-2959-4625-8aed-b7d39c5e9d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:31.462816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5c1c6-2959-4625-8aed-b7d39c5e9d38 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:31.463694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e5c1c6-2959-4625-8aed-b7d39c5e9d38 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:31.464559 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e5c1c6-2959-4625-8aed-b7d39c5e9d38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:31.466077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e5c1c6-2959-4625-8aed-b7d39c5e9d38 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:31.493790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 383/751] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jun 17 07:07:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:31.493790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5c1c6-2959-4625-8aed-b7d39c5e9d38 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:31.502424 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec78d4-0891-4f30-b2f0-1c502b5850b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:31.503234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec78d4-0891-4f30-b2f0-1c502b5850b3 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:31.503519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec78d4-0891-4f30-b2f0-1c502b5850b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:31.503824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec78d4-0891-4f30-b2f0-1c502b5850b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:31.513959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec78d4-0891-4f30-b2f0-1c502b5850b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:31.515421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec78d4-0891-4f30-b2f0-1c502b5850b3 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-d4ec78d4-0891-4f30-b2f0-1c502b5850b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:31.532743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:31.533675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:31.534596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:31.534596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:31.534596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:31.534596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:31.534596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:31.534596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:31.534596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:31.534596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:31.534596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 369/752] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:31.535078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec78d4-0891-4f30-b2f0-1c502b5850b3 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:31.611495 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5c1c6-2959-4625-8aed-b7d39c5e9d38 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:31.640730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:31.653682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec78d4-0891-4f30-b2f0-1c502b5850b3 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:31.662451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd061f-4cef-40e2-b0b2-9aba483279cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:31.663188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd061f-4cef-40e2-b0b2-9aba483279cd None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:31.663477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd061f-4cef-40e2-b0b2-9aba483279cd None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:31.663753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd061f-4cef-40e2-b0b2-9aba483279cd None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:31.664980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd061f-4cef-40e2-b0b2-9aba483279cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:31.665354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd061f-4cef-40e2-b0b2-9aba483279cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:31.667942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd061f-4cef-40e2-b0b2-9aba483279cd None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:31.693516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 384/753] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jun 17 07:07:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:31.694431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd061f-4cef-40e2-b0b2-9aba483279cd None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:31.711430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9df390-8d71-47bc-81e8-7d78997ae14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:31.712698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df390-8d71-47bc-81e8-7d78997ae14d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:31.712992 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df390-8d71-47bc-81e8-7d78997ae14d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:31.714192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df390-8d71-47bc-81e8-7d78997ae14d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:31.728625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9df390-8d71-47bc-81e8-7d78997ae14d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:31.730966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9df390-8d71-47bc-81e8-7d78997ae14d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:31.738484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-8f9df390-8d71-47bc-81e8-7d78997ae14d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:31.756245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:31.757338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:31.758413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:31.758413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:31.758413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:31.758413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:31.761716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 370/754] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:31.763464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9df390-8d71-47bc-81e8-7d78997ae14d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:31.812035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd061f-4cef-40e2-b0b2-9aba483279cd None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:31.864508 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8a5c1-2997-4951-bfa9-e6f105b973de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:31.865977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8a5c1-2997-4951-bfa9-e6f105b973de None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:31.866663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8a5c1-2997-4951-bfa9-e6f105b973de None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:31.867099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8a5c1-2997-4951-bfa9-e6f105b973de None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:31.868333 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8a5c1-2997-4951-bfa9-e6f105b973de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:31.869183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8a5c1-2997-4951-bfa9-e6f105b973de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:31.870819 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8a5c1-2997-4951-bfa9-e6f105b973de None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:31.886558 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9df390-8d71-47bc-81e8-7d78997ae14d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:31.890190 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 385/755] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:31.891327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8a5c1-2997-4951-bfa9-e6f105b973de None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:31.933638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454e95-98cb-4032-8603-ce23ed931988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:31.934694 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454e95-98cb-4032-8603-ce23ed931988 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:31.935187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454e95-98cb-4032-8603-ce23ed931988 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:31.935669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454e95-98cb-4032-8603-ce23ed931988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:31.954297 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:31.997506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:32.004851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8a5c1-2997-4951-bfa9-e6f105b973de None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:32.039065 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3454e95-98cb-4032-8603-ce23ed931988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:32.041113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3454e95-98cb-4032-8603-ce23ed931988 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:32.041226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12ce23-dea0-432c-9bca-249bf8214e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:32.041881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ce23-dea0-432c-9bca-249bf8214e47 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:32.042169 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ce23-dea0-432c-9bca-249bf8214e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:32.042446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ce23-dea0-432c-9bca-249bf8214e47 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:32.043165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12ce23-dea0-432c-9bca-249bf8214e47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:32.043472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12ce23-dea0-432c-9bca-249bf8214e47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:32.044868 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12ce23-dea0-432c-9bca-249bf8214e47 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:32.057999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 386/756] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:32.058910 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12ce23-dea0-432c-9bca-249bf8214e47 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:32.173322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12ce23-dea0-432c-9bca-249bf8214e47 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:32.174830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 371/757] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:07:32.175904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454e95-98cb-4032-8603-ce23ed931988 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:32.212731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20947e7-7bc3-4b7c-b252-b6a30c4ed7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:32.214597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20947e7-7bc3-4b7c-b252-b6a30c4ed7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:32.214597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20947e7-7bc3-4b7c-b252-b6a30c4ed7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:32.214597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20947e7-7bc3-4b7c-b252-b6a30c4ed7e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:32.215295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20947e7-7bc3-4b7c-b252-b6a30c4ed7e4 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=004e5909f9834cfea890063bb0083d7c) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:32.215607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20947e7-7bc3-4b7c-b252-b6a30c4ed7e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:32.217052 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20947e7-7bc3-4b7c-b252-b6a30c4ed7e4 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:32.224283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:32.228769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 387/758] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:32 2022] GET /identity/v3/role_assignments?user.id=004e5909f9834cfea890063bb0083d7c&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:32.230602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20947e7-7bc3-4b7c-b252-b6a30c4ed7e4 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:32.269388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:32.292448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454e95-98cb-4032-8603-ce23ed931988 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:32.332292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c5092-2367-4bfe-90d1-87562d40226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:32.333715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c5092-2367-4bfe-90d1-87562d40226b None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:32.334237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c5092-2367-4bfe-90d1-87562d40226b None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:32.334756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c5092-2367-4bfe-90d1-87562d40226b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:32.353788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c5092-2367-4bfe-90d1-87562d40226b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:32.357294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c5092-2367-4bfe-90d1-87562d40226b None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-0a7c5092-2367-4bfe-90d1-87562d40226b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:32.381573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:32.382566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:32.383478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:32.383478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:32.383478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:32.383478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:32.383478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:32.383478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:32.383478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:32.383478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:32.390415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 372/759] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:32.395111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c5092-2367-4bfe-90d1-87562d40226b None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:32.433911 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20947e7-7bc3-4b7c-b252-b6a30c4ed7e4 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:32.502598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ee8f6-e015-4fa8-a5fd-c3df74fb3017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:32.503412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ee8f6-e015-4fa8-a5fd-c3df74fb3017 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:32.503697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ee8f6-e015-4fa8-a5fd-c3df74fb3017 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:32.503976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ee8f6-e015-4fa8-a5fd-c3df74fb3017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:32.527992 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ee8f6-e015-4fa8-a5fd-c3df74fb3017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:32.529502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ee8f6-e015-4fa8-a5fd-c3df74fb3017 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-861ee8f6-e015-4fa8-a5fd-c3df74fb3017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:32.557482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:32.558457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:32.559369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:32.559369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:32.559369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:32.559369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:32.559369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 388/760] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:32.569680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ee8f6-e015-4fa8-a5fd-c3df74fb3017 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:32.591702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c5092-2367-4bfe-90d1-87562d40226b None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:32.593290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:32.658014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ade9c0-9190-4644-8032-562e9bb1ea93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:32.658675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ade9c0-9190-4644-8032-562e9bb1ea93 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:32.658967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ade9c0-9190-4644-8032-562e9bb1ea93 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:32.659247 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ade9c0-9190-4644-8032-562e9bb1ea93 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:32.660138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ade9c0-9190-4644-8032-562e9bb1ea93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:32.660453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ade9c0-9190-4644-8032-562e9bb1ea93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:32.661956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ade9c0-9190-4644-8032-562e9bb1ea93 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:32.688308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 373/761] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:32.694201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ade9c0-9190-4644-8032-562e9bb1ea93 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:32.709635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:32.742610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ee8f6-e015-4fa8-a5fd-c3df74fb3017 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:32.780172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce93a57-35b8-4a06-bf86-9d2a31026b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:32.780894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce93a57-35b8-4a06-bf86-9d2a31026b49 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:32.781182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce93a57-35b8-4a06-bf86-9d2a31026b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:32.781459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce93a57-35b8-4a06-bf86-9d2a31026b49 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:32.782225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce93a57-35b8-4a06-bf86-9d2a31026b49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:32.782659 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce93a57-35b8-4a06-bf86-9d2a31026b49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:32.784152 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce93a57-35b8-4a06-bf86-9d2a31026b49 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:32.799475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 389/762] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:32.800786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f2a01-a816-44a2-9ebb-92dc590bcfe4 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:32.801076 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f2a01-a816-44a2-9ebb-92dc590bcfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:32.801357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f2a01-a816-44a2-9ebb-92dc590bcfe4 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:32.802569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 390/763] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:32.805704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f2a01-a816-44a2-9ebb-92dc590bcfe4 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:32.826688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ade9c0-9190-4644-8032-562e9bb1ea93 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:32.862550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f46eb-17cf-4404-b1d1-6f5f17eb53ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:32.863439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f46eb-17cf-4404-b1d1-6f5f17eb53ff None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:32.863725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f46eb-17cf-4404-b1d1-6f5f17eb53ff None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:32.864006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f46eb-17cf-4404-b1d1-6f5f17eb53ff None admin] PATH_INFO: `/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/8d3baa16824f40f3a4c783e210b262f4/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:32.866095 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f46eb-17cf-4404-b1d1-6f5f17eb53ff None admin] RBAC: Authorizing `identity:create_grant(project_id=a4ec9b09302d4c918ef638d794468ba3, user_id=8d3baa16824f40f3a4c783e210b262f4, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:32.867549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f46eb-17cf-4404-b1d1-6f5f17eb53ff None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:32.888224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 374/764] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:07:32 2022] PUT /identity/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/8d3baa16824f40f3a4c783e210b262f4/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:07:32.889695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee66665-af81-4a99-ad48-0dac5e27b205 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:32.889988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee66665-af81-4a99-ad48-0dac5e27b205 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:32.890334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee66665-af81-4a99-ad48-0dac5e27b205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:32.920322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f2a01-a816-44a2-9ebb-92dc590bcfe4 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:32.923553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-8ee66665-af81-4a99-ad48-0dac5e27b205 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:32.924849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8ee66665-af81-4a99-ad48-0dac5e27b205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:32.952751 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:32.962419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:32.972897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e83d98-c797-415c-adaf-92f604323ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:32.973850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e83d98-c797-415c-adaf-92f604323ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:32.974186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e83d98-c797-415c-adaf-92f604323ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:32.974467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e83d98-c797-415c-adaf-92f604323ba1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:32.985019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e83d98-c797-415c-adaf-92f604323ba1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:32.986953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e83d98-c797-415c-adaf-92f604323ba1 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-21e83d98-c797-415c-adaf-92f604323ba1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:33.004963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:33.007118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:33.009340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:33.009340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:33.009340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:33.009340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:33.009340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:33.009340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:33.009340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:33.009340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:33.016349 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 391/765] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:33.017933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e83d98-c797-415c-adaf-92f604323ba1 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:33.143522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 375/766] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:33.144895 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e83d98-c797-415c-adaf-92f604323ba1 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:33.182525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee66665-af81-4a99-ad48-0dac5e27b205 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:33.185338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36b490-ddc5-4d3d-aee6-73e50fc6cc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:33.186951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36b490-ddc5-4d3d-aee6-73e50fc6cc68 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:33.187240 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36b490-ddc5-4d3d-aee6-73e50fc6cc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:33.187516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36b490-ddc5-4d3d-aee6-73e50fc6cc68 None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:33.188383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36b490-ddc5-4d3d-aee6-73e50fc6cc68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:33.188693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36b490-ddc5-4d3d-aee6-73e50fc6cc68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:33.190184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36b490-ddc5-4d3d-aee6-73e50fc6cc68 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:33.212233 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 392/767] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:33.215710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36b490-ddc5-4d3d-aee6-73e50fc6cc68 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:33.283376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:33.306668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee66665-af81-4a99-ad48-0dac5e27b205 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:33.346298 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d5e4e-6700-4ab1-adbb-3eff4586cc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:33.347103 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d5e4e-6700-4ab1-adbb-3eff4586cc2f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:33.347777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d5e4e-6700-4ab1-adbb-3eff4586cc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:33.348087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d5e4e-6700-4ab1-adbb-3eff4586cc2f None admin] PATH_INFO: `/v3/services/network` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:33.348741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d5e4e-6700-4ab1-adbb-3eff4586cc2f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:33.350177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d5e4e-6700-4ab1-adbb-3eff4586cc2f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-242d5e4e-6700-4ab1-adbb-3eff4586cc2f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:33.364846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:33.367119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 07:07:33.369175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:33.370989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 376/768] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:33 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:33.376095 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d5e4e-6700-4ab1-adbb-3eff4586cc2f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:33.393022 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:33.422047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36b490-ddc5-4d3d-aee6-73e50fc6cc68 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:33.487492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d5e4e-6700-4ab1-adbb-3eff4586cc2f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:33.504900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad4c1b-d171-4b62-9b3d-baf0d545df85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:33.505597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad4c1b-d171-4b62-9b3d-baf0d545df85 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:33.505887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad4c1b-d171-4b62-9b3d-baf0d545df85 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:33.506165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad4c1b-d171-4b62-9b3d-baf0d545df85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:33.507048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad4c1b-d171-4b62-9b3d-baf0d545df85 None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=dc98526f14e84279b002c43ef44cabed) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:33.507350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad4c1b-d171-4b62-9b3d-baf0d545df85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:33.508992 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad4c1b-d171-4b62-9b3d-baf0d545df85 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:33.515714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 393/769] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:33 2022] GET /identity/v3/role_assignments?user.id=dc98526f14e84279b002c43ef44cabed&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:33.539008 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a022f-04b2-4c3e-a39e-29318006903e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:33.539697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a022f-04b2-4c3e-a39e-29318006903e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:33.539979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a022f-04b2-4c3e-a39e-29318006903e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:33.540284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a022f-04b2-4c3e-a39e-29318006903e None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:33.541016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a022f-04b2-4c3e-a39e-29318006903e None admin] RBAC: Adding query filter params (name=network) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:33.541357 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a022f-04b2-4c3e-a39e-29318006903e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:33.542742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a022f-04b2-4c3e-a39e-29318006903e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:33.552175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 377/770] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:07:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:33.554224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad4c1b-d171-4b62-9b3d-baf0d545df85 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:33.713797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66164235-be24-4bb8-82f0-0bba30f2ebf0 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:33.714450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66164235-be24-4bb8-82f0-0bba30f2ebf0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:33.714450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66164235-be24-4bb8-82f0-0bba30f2ebf0 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:33.719920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 378/771] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:33.721800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ba8dc-1d32-4ec8-a0da-33b5e390d85c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:33.722092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ba8dc-1d32-4ec8-a0da-33b5e390d85c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:33.722370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ba8dc-1d32-4ec8-a0da-33b5e390d85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:33.741507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-3e1ba8dc-1d32-4ec8-a0da-33b5e390d85c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:33.742659 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3e1ba8dc-1d32-4ec8-a0da-33b5e390d85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:33.821221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad4c1b-d171-4b62-9b3d-baf0d545df85 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:33.828414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 379/772] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:33.840323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:33.848642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ba8dc-1d32-4ec8-a0da-33b5e390d85c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:33.861824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:33.862823 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6eb55-a034-4ae2-89b2-bcb406826afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:33.863498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6eb55-a034-4ae2-89b2-bcb406826afb None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:33.863949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6eb55-a034-4ae2-89b2-bcb406826afb None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:33.864599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6eb55-a034-4ae2-89b2-bcb406826afb None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:33.865322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6eb55-a034-4ae2-89b2-bcb406826afb None admin] RBAC: Adding query filter params (type=network) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:33.865783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6eb55-a034-4ae2-89b2-bcb406826afb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:33.867691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6eb55-a034-4ae2-89b2-bcb406826afb None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:33.874679 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 394/773] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:07:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:33.886712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6eb55-a034-4ae2-89b2-bcb406826afb None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:34.027684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6eb55-a034-4ae2-89b2-bcb406826afb None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:34.057360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ba8dc-1d32-4ec8-a0da-33b5e390d85c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:34.094205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fb9cd-b68d-42e8-bf3f-5f4d8131e456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:34.095021 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fb9cd-b68d-42e8-bf3f-5f4d8131e456 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:34.095306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fb9cd-b68d-42e8-bf3f-5f4d8131e456 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:34.095582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fb9cd-b68d-42e8-bf3f-5f4d8131e456 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:34.096258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fb9cd-b68d-42e8-bf3f-5f4d8131e456 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:34.097343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66436af7-8349-4cc2-aacb-789a4c1e1bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:34.097752 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fb9cd-b68d-42e8-bf3f-5f4d8131e456 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:34.098013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66436af7-8349-4cc2-aacb-789a4c1e1bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:34.098300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66436af7-8349-4cc2-aacb-789a4c1e1bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:34.098576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66436af7-8349-4cc2-aacb-789a4c1e1bb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:34.099375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66436af7-8349-4cc2-aacb-789a4c1e1bb7 None admin] RBAC: Adding query filter params (interface=public, service_id=662592c3f734479e9c1789228bccc844, region_id=RegionOne) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:34.099690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66436af7-8349-4cc2-aacb-789a4c1e1bb7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:34.101075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66436af7-8349-4cc2-aacb-789a4c1e1bb7 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-785fb9cd-b68d-42e8-bf3f-5f4d8131e456 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:34.111013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:34.111968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 07:07:34.113274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:34.116184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 380/774] 10.222.0.33 () {56 vars in 1129 bytes} [Fri Jun 17 07:07:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:34.118548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fb9cd-b68d-42e8-bf3f-5f4d8131e456 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:34.120489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 395/775] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jun 17 07:07:33 2022] GET /identity/v3/endpoints?service_id=662592c3f734479e9c1789228bccc844&interface=public®ion_id=RegionOne => generated 194 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:34.132479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66436af7-8349-4cc2-aacb-789a4c1e1bb7 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:34.180577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:34.263427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:34.268704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66436af7-8349-4cc2-aacb-789a4c1e1bb7 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:34.292999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fb9cd-b68d-42e8-bf3f-5f4d8131e456 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:34.304047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9862fd-8326-4799-abe7-eb0e506a3100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:34.304743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9862fd-8326-4799-abe7-eb0e506a3100 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:34.305029 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9862fd-8326-4799-abe7-eb0e506a3100 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:34.305306 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9862fd-8326-4799-abe7-eb0e506a3100 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:34.305987 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9862fd-8326-4799-abe7-eb0e506a3100 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:34.307412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9862fd-8326-4799-abe7-eb0e506a3100 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:34.320875 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 396/776] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jun 17 07:07:34 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:34.331388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0e795-39fd-41b4-a454-8dcbba4c6a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:34.332099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0e795-39fd-41b4-a454-8dcbba4c6a62 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:34.332375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0e795-39fd-41b4-a454-8dcbba4c6a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:34.332708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0e795-39fd-41b4-a454-8dcbba4c6a62 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:34.333429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0e795-39fd-41b4-a454-8dcbba4c6a62 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:34.333787 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0e795-39fd-41b4-a454-8dcbba4c6a62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:34.335160 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0e795-39fd-41b4-a454-8dcbba4c6a62 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:34.344212 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 381/777] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jun 17 07:07:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:34.349263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9862fd-8326-4799-abe7-eb0e506a3100 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:34.490073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9862fd-8326-4799-abe7-eb0e506a3100 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:34.512479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:34.533229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25605808-2a69-4943-a427-545588565b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:34.533896 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25605808-2a69-4943-a427-545588565b74 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:34.534182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25605808-2a69-4943-a427-545588565b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:34.534459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25605808-2a69-4943-a427-545588565b74 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:34.535165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25605808-2a69-4943-a427-545588565b74 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:34.535471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25605808-2a69-4943-a427-545588565b74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:34.536856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25605808-2a69-4943-a427-545588565b74 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:34.542573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 397/778] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jun 17 07:07:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:34.548196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0e795-39fd-41b4-a454-8dcbba4c6a62 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:34.661865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0e795-39fd-41b4-a454-8dcbba4c6a62 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:34.687455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76822eea-5824-47a7-82da-3d26d5156bb0 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:34.687856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76822eea-5824-47a7-82da-3d26d5156bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:34.688020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76822eea-5824-47a7-82da-3d26d5156bb0 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:34.689588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 398/779] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:34.694893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f9c48-c7a1-4a8c-9a67-7100929b78fe None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:34.695445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f9c48-c7a1-4a8c-9a67-7100929b78fe None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:34.695959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f9c48-c7a1-4a8c-9a67-7100929b78fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:34.700097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544617e-2a4d-4545-8109-2d1f1c1f19a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:34.700097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544617e-2a4d-4545-8109-2d1f1c1f19a7 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:34.700097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544617e-2a4d-4545-8109-2d1f1c1f19a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:34.700097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544617e-2a4d-4545-8109-2d1f1c1f19a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:34.700439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544617e-2a4d-4545-8109-2d1f1c1f19a7 None admin] RBAC: Adding query filter params (interface=public, service_id=370ab0706a1141b1b8ebc4453f528fe5, region_id=RegionOne) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:34.700662 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544617e-2a4d-4545-8109-2d1f1c1f19a7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:34.704137 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544617e-2a4d-4545-8109-2d1f1c1f19a7 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:34.715303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 382/780] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jun 17 07:07:34 2022] GET /identity/v3/endpoints?service_id=370ab0706a1141b1b8ebc4453f528fe5&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:34.717171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544617e-2a4d-4545-8109-2d1f1c1f19a7 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:34.719905 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:34.737076 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-470f9c48-c7a1-4a8c-9a67-7100929b78fe None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:34.738607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-470f9c48-c7a1-4a8c-9a67-7100929b78fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:34.825174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544617e-2a4d-4545-8109-2d1f1c1f19a7 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:34.860495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36e83f-9570-4595-be69-be5c649d4559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:34.861160 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36e83f-9570-4595-be69-be5c649d4559 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:34.861445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36e83f-9570-4595-be69-be5c649d4559 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:34.861723 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36e83f-9570-4595-be69-be5c649d4559 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:34.862398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36e83f-9570-4595-be69-be5c649d4559 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:34.863771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36e83f-9570-4595-be69-be5c649d4559 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:34.874339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 383/781] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jun 17 07:07:34 2022] GET /identity/v3/services => generated 772 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:34.876959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 399/782] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:34.912433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36e83f-9570-4595-be69-be5c649d4559 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:35.000900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:35.051186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36e83f-9570-4595-be69-be5c649d4559 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:35.105588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c808a1-77e6-4919-847a-e06f04dd6654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:35.106324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c808a1-77e6-4919-847a-e06f04dd6654 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:35.106614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c808a1-77e6-4919-847a-e06f04dd6654 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:35.106896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c808a1-77e6-4919-847a-e06f04dd6654 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:35.117987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c808a1-77e6-4919-847a-e06f04dd6654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-47c808a1-77e6-4919-847a-e06f04dd6654 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:35.126406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:35.127364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:35.128250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 384/783] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:07:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:35.133680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f9c48-c7a1-4a8c-9a67-7100929b78fe None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:35.243326 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f9c48-c7a1-4a8c-9a67-7100929b78fe None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:35.244988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:35.280670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8d913-9491-45fd-ae76-29d613f0bc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:35.281343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8d913-9491-45fd-ae76-29d613f0bc77 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:35.281624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8d913-9491-45fd-ae76-29d613f0bc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:35.281900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8d913-9491-45fd-ae76-29d613f0bc77 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:35.282645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8d913-9491-45fd-ae76-29d613f0bc77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:35.282952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8d913-9491-45fd-ae76-29d613f0bc77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:35.284329 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8d913-9491-45fd-ae76-29d613f0bc77 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:35.293519 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 400/784] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:35.298953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c808a1-77e6-4919-847a-e06f04dd6654 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:35.433784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c808a1-77e6-4919-847a-e06f04dd6654 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:35.474003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:35.484496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35977a-8739-4e25-a905-455bfa7374cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:35.485304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35977a-8739-4e25-a905-455bfa7374cf None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:35.485592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35977a-8739-4e25-a905-455bfa7374cf None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:35.485870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35977a-8739-4e25-a905-455bfa7374cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:35.500310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35977a-8739-4e25-a905-455bfa7374cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:35.502083 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35977a-8739-4e25-a905-455bfa7374cf None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-df35977a-8739-4e25-a905-455bfa7374cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:35.519663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:35.520635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:35.521551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:35.521551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:35.521551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:35.521551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:35.523824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 385/785] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:35.533189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8d913-9491-45fd-ae76-29d613f0bc77 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:35.621245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac80c7-66e3-4c9e-a3d2-cda617ee96ff None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:35.621882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac80c7-66e3-4c9e-a3d2-cda617ee96ff None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:35.622391 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac80c7-66e3-4c9e-a3d2-cda617ee96ff None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:35.624284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 386/786] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:35.629467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bde62-3dce-4d27-9ce3-8f4f779ffcf7 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:35.630002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bde62-3dce-4d27-9ce3-8f4f779ffcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:35.630488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bde62-3dce-4d27-9ce3-8f4f779ffcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:35.651413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-e45bde62-3dce-4d27-9ce3-8f4f779ffcf7 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:35.657912 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e45bde62-3dce-4d27-9ce3-8f4f779ffcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:35.701113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8d913-9491-45fd-ae76-29d613f0bc77 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:35.769582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c427b-866a-4532-84f8-4c4bb84eb9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:35.770638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c427b-866a-4532-84f8-4c4bb84eb9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:35.771133 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c427b-866a-4532-84f8-4c4bb84eb9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:35.771640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c427b-866a-4532-84f8-4c4bb84eb9c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:35.772611 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c427b-866a-4532-84f8-4c4bb84eb9c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:35.773165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c427b-866a-4532-84f8-4c4bb84eb9c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:35.776565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c427b-866a-4532-84f8-4c4bb84eb9c0 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:35.787146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 387/787] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:35.799493 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 401/788] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:35.804622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bde62-3dce-4d27-9ce3-8f4f779ffcf7 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:35.875467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c427b-866a-4532-84f8-4c4bb84eb9c0 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:35.932424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:35.967395 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bde62-3dce-4d27-9ce3-8f4f779ffcf7 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:35.988438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:36.022340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba13e3c-0165-4b49-a43f-3e4444569936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:36.023503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba13e3c-0165-4b49-a43f-3e4444569936 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:36.023998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba13e3c-0165-4b49-a43f-3e4444569936 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:36.024595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba13e3c-0165-4b49-a43f-3e4444569936 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:36.033180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba13e3c-0165-4b49-a43f-3e4444569936 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:36.034926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba13e3c-0165-4b49-a43f-3e4444569936 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-eba13e3c-0165-4b49-a43f-3e4444569936 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:36.050609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:36.051579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:36.052593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:36.052593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:36.052593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:36.052593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:36.052593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:36.052593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:36.052593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:36.052593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:36.052593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 07:07:36.052593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:36.054944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 388/789] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:07:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:36.061462 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba13e3c-0165-4b49-a43f-3e4444569936 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:36.077277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c427b-866a-4532-84f8-4c4bb84eb9c0 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:36.168802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad29d4-5070-4711-a97f-203a265598ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:36.169618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad29d4-5070-4711-a97f-203a265598ed None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:36.169908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad29d4-5070-4711-a97f-203a265598ed None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:36.170184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad29d4-5070-4711-a97f-203a265598ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:36.190779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad29d4-5070-4711-a97f-203a265598ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:36.199075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba13e3c-0165-4b49-a43f-3e4444569936 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-2dad29d4-5070-4711-a97f-203a265598ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:36.199400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:36.200430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:36.201277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 402/790] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:07:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:36.202463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e98f2a-fd0b-49bb-a02e-5779f4abd82c None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:36.202463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e98f2a-fd0b-49bb-a02e-5779f4abd82c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:36.202697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e98f2a-fd0b-49bb-a02e-5779f4abd82c None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:36.208715 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 403/791] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:36.209500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e98f2a-fd0b-49bb-a02e-5779f4abd82c None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:36.239964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e09a3-eed4-4727-9775-f0bf0edf50ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:36.241052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e09a3-eed4-4727-9775-f0bf0edf50ee None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:36.241584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e09a3-eed4-4727-9775-f0bf0edf50ee None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:36.242069 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e09a3-eed4-4727-9775-f0bf0edf50ee None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:36.243113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e09a3-eed4-4727-9775-f0bf0edf50ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:36.243750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e09a3-eed4-4727-9775-f0bf0edf50ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:36.245506 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e09a3-eed4-4727-9775-f0bf0edf50ee None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:36.264383 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 389/792] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jun 17 07:07:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:36.266003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694da86-e919-4965-8cd9-42c6544a1e48 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:36.266523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694da86-e919-4965-8cd9-42c6544a1e48 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:36.267061 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694da86-e919-4965-8cd9-42c6544a1e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:36.284545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-a694da86-e919-4965-8cd9-42c6544a1e48 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:36.286086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-a694da86-e919-4965-8cd9-42c6544a1e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:36.305026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:36.308562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:36.327421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e98f2a-fd0b-49bb-a02e-5779f4abd82c None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:36.360794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 390/793] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:36.362107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694da86-e919-4965-8cd9-42c6544a1e48 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:36.364537 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c808ddf-5c9d-4f28-b9ae-67e33caa0aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:36.365321 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c808ddf-5c9d-4f28-b9ae-67e33caa0aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:36.365614 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c808ddf-5c9d-4f28-b9ae-67e33caa0aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:36.365892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c808ddf-5c9d-4f28-b9ae-67e33caa0aa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:36.367220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c808ddf-5c9d-4f28-b9ae-67e33caa0aa1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:36.367220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c808ddf-5c9d-4f28-b9ae-67e33caa0aa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:36.368418 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c808ddf-5c9d-4f28-b9ae-67e33caa0aa1 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:36.377550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 404/794] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:36.382851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c808ddf-5c9d-4f28-b9ae-67e33caa0aa1 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:36.481878 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694da86-e919-4965-8cd9-42c6544a1e48 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:36.497487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c808ddf-5c9d-4f28-b9ae-67e33caa0aa1 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:36.519921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a9bb6-bb70-4377-acc6-fdb13686e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:36.520812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a9bb6-bb70-4377-acc6-fdb13686e40b None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:36.521321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a9bb6-bb70-4377-acc6-fdb13686e40b None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:36.521877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a9bb6-bb70-4377-acc6-fdb13686e40b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:36.533029 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c7fec-3b07-4f97-b0fe-372680b8292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:36.533345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a9bb6-bb70-4377-acc6-fdb13686e40b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:36.534197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c7fec-3b07-4f97-b0fe-372680b8292d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:36.534630 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c7fec-3b07-4f97-b0fe-372680b8292d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:36.534878 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a9bb6-bb70-4377-acc6-fdb13686e40b None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:36.535212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c7fec-3b07-4f97-b0fe-372680b8292d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:36.546965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c7fec-3b07-4f97-b0fe-372680b8292d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:36.548621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c7fec-3b07-4f97-b0fe-372680b8292d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-6c1a9bb6-bb70-4377-acc6-fdb13686e40b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:36.553850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:36.554908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:36.556000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:36.556000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:36.556000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:36.556000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:36.556757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:36.559266 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 391/795] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:36.560407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a9bb6-bb70-4377-acc6-fdb13686e40b None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-110c7fec-3b07-4f97-b0fe-372680b8292d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:36.571882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:36.573020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:36.573923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:36.573923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:36.573923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:36.573923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:36.578240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:36.578373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 405/796] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:36.579936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c7fec-3b07-4f97-b0fe-372680b8292d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:36.674749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a9bb6-bb70-4377-acc6-fdb13686e40b None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:36.687592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c7fec-3b07-4f97-b0fe-372680b8292d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:36.713376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f3e7-5f64-4700-a790-15dd8c473ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:36.714481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f3e7-5f64-4700-a790-15dd8c473ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:36.714963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f3e7-5f64-4700-a790-15dd8c473ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:36.715439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f3e7-5f64-4700-a790-15dd8c473ec1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:36.723971 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730136b-28f9-49ff-a65f-bf2ce39faf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:36.724665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730136b-28f9-49ff-a65f-bf2ce39faf2a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:36.724955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730136b-28f9-49ff-a65f-bf2ce39faf2a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:36.725234 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730136b-28f9-49ff-a65f-bf2ce39faf2a None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:36.725959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730136b-28f9-49ff-a65f-bf2ce39faf2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:36.726270 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730136b-28f9-49ff-a65f-bf2ce39faf2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:36.727658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730136b-28f9-49ff-a65f-bf2ce39faf2a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:36.727996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6f3e7-5f64-4700-a790-15dd8c473ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-f4c6f3e7-5f64-4700-a790-15dd8c473ec1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:36.737283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:36.738330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:36.739293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:36.739293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:36.739293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:36.739293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:36.739293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:36.739293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:36.739293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:36.739293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:36.739293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:36.739293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:36.739293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:36.739293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:36.741514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 392/797] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:36.742441 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f3e7-5f64-4700-a790-15dd8c473ec1 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:36.745048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 406/798] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:36.745938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730136b-28f9-49ff-a65f-bf2ce39faf2a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:36.808227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:36.819455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:36.854817 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730136b-28f9-49ff-a65f-bf2ce39faf2a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:36.857287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f3e7-5f64-4700-a790-15dd8c473ec1 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:36.891207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00571b94-eb46-4f35-9990-fc87cd4fef39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:36.892013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00571b94-eb46-4f35-9990-fc87cd4fef39 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:36.892317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00571b94-eb46-4f35-9990-fc87cd4fef39 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:36.892601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00571b94-eb46-4f35-9990-fc87cd4fef39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:36.895490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4ef3e-4e7f-4e37-8539-e62dea83f7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:36.896418 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4ef3e-4e7f-4e37-8539-e62dea83f7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:36.896919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4ef3e-4e7f-4e37-8539-e62dea83f7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:36.897467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4ef3e-4e7f-4e37-8539-e62dea83f7b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:36.898419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4ef3e-4e7f-4e37-8539-e62dea83f7b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:36.899046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4ef3e-4e7f-4e37-8539-e62dea83f7b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:36.899476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00571b94-eb46-4f35-9990-fc87cd4fef39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:36.901710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4ef3e-4e7f-4e37-8539-e62dea83f7b5 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:36.901826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00571b94-eb46-4f35-9990-fc87cd4fef39 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:36.915068 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 393/799] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:36.916309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4ef3e-4e7f-4e37-8539-e62dea83f7b5 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-00571b94-eb46-4f35-9990-fc87cd4fef39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:36.920638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:36.921525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:36.922438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:36.922438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:36.922438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:36.922438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:36.922438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:36.922438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:36.922438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:36.922438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:36.922438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 407/800] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:36.922937 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00571b94-eb46-4f35-9990-fc87cd4fef39 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:37.028124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4ef3e-4e7f-4e37-8539-e62dea83f7b5 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:37.031515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00571b94-eb46-4f35-9990-fc87cd4fef39 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:37.049209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:37.067055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c02b8-ae2c-4d55-ab7f-008295f57f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:37.067878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c02b8-ae2c-4d55-ab7f-008295f57f69 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.068194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c02b8-ae2c-4d55-ab7f-008295f57f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.068474 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c02b8-ae2c-4d55-ab7f-008295f57f69 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.071848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565f898-92a4-4202-880a-42583043fd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:37.073018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565f898-92a4-4202-880a-42583043fd6d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.073527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565f898-92a4-4202-880a-42583043fd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.074024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565f898-92a4-4202-880a-42583043fd6d None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.075087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565f898-92a4-4202-880a-42583043fd6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:37.075632 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565f898-92a4-4202-880a-42583043fd6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:37.077438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c02b8-ae2c-4d55-ab7f-008295f57f69 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:37.078900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c02b8-ae2c-4d55-ab7f-008295f57f69 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:37.080294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:37.080927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565f898-92a4-4202-880a-42583043fd6d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:37.091245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 394/801] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:37.092575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565f898-92a4-4202-880a-42583043fd6d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-122c02b8-ae2c-4d55-ab7f-008295f57f69 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:37.095622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:37.096594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:37.097543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:37.097543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:37.097543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:37.097543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:37.097543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:37.097543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:37.097543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:37.097543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:37.097543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:07:37.097543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:37.101937 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 408/802] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jun 17 07:07:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:37.102954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c02b8-ae2c-4d55-ab7f-008295f57f69 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:37.211109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565f898-92a4-4202-880a-42583043fd6d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:37.212296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c02b8-ae2c-4d55-ab7f-008295f57f69 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:37.247182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a41c6-3ff9-4b81-959d-f03d6ee11a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:37.247981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a41c6-3ff9-4b81-959d-f03d6ee11a30 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.248299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a41c6-3ff9-4b81-959d-f03d6ee11a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.248642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a41c6-3ff9-4b81-959d-f03d6ee11a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.251965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dcb7e-1efb-4802-b4a7-fe74ea871500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:37.252896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dcb7e-1efb-4802-b4a7-fe74ea871500 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.253333 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dcb7e-1efb-4802-b4a7-fe74ea871500 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.253762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dcb7e-1efb-4802-b4a7-fe74ea871500 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.254717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dcb7e-1efb-4802-b4a7-fe74ea871500 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:37.255179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dcb7e-1efb-4802-b4a7-fe74ea871500 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:37.257152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dcb7e-1efb-4802-b4a7-fe74ea871500 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:37.259096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a41c6-3ff9-4b81-959d-f03d6ee11a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:37.261088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a41c6-3ff9-4b81-959d-f03d6ee11a30 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:37.275631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 395/803] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:37.275631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dcb7e-1efb-4802-b4a7-fe74ea871500 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-a99a41c6-3ff9-4b81-959d-f03d6ee11a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:37.278325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:37.279186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:37.280120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:37.280120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:37.280120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:37.280120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:37.281625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 409/804] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:37.282025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc2b55-3cee-4069-b96b-35b0272fb566 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.282302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc2b55-3cee-4069-b96b-35b0272fb566 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.282580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc2b55-3cee-4069-b96b-35b0272fb566 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.284284 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 410/805] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:37.284972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc2b55-3cee-4069-b96b-35b0272fb566 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:37.319683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:37.342465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:37.393659 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dcb7e-1efb-4802-b4a7-fe74ea871500 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:37.398402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc2b55-3cee-4069-b96b-35b0272fb566 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:37.428087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6299fa-529b-4265-b351-647b0544ebbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:37.428764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6299fa-529b-4265-b351-647b0544ebbc None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.429052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6299fa-529b-4265-b351-647b0544ebbc None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.429334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6299fa-529b-4265-b351-647b0544ebbc None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.430166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6299fa-529b-4265-b351-647b0544ebbc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:37.430479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6299fa-529b-4265-b351-647b0544ebbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:37.431948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6299fa-529b-4265-b351-647b0544ebbc None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:37.432736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e6ecb-87eb-4854-9619-1cc044a4b183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:37.433620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e6ecb-87eb-4854-9619-1cc044a4b183 None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.433913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e6ecb-87eb-4854-9619-1cc044a4b183 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.434188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e6ecb-87eb-4854-9619-1cc044a4b183 None admin] PATH_INFO: `/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/004e5909f9834cfea890063bb0083d7c/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.436153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e6ecb-87eb-4854-9619-1cc044a4b183 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4ec9b09302d4c918ef638d794468ba3, user_id=004e5909f9834cfea890063bb0083d7c, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:37.437574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e6ecb-87eb-4854-9619-1cc044a4b183 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:37.448761 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 396/806] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jun 17 07:07:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:37.449562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6299fa-529b-4265-b351-647b0544ebbc None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:37.450070 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 411/807] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:07:37 2022] PUT /identity/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/004e5909f9834cfea890063bb0083d7c/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:07:37.451370 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6efca-90b2-4224-a2b7-08864658eb43 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.451668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6efca-90b2-4224-a2b7-08864658eb43 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.451948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6efca-90b2-4224-a2b7-08864658eb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.470552 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-89f6efca-90b2-4224-a2b7-08864658eb43 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:37.471787 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-89f6efca-90b2-4224-a2b7-08864658eb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:37.554396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 412/808] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:37.557110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adab2b-f39e-4ea3-9a9c-ab6eb9a7c206 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.557607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adab2b-f39e-4ea3-9a9c-ab6eb9a7c206 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.558243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adab2b-f39e-4ea3-9a9c-ab6eb9a7c206 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.564661 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 413/809] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:37.565357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adab2b-f39e-4ea3-9a9c-ab6eb9a7c206 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:37.575752 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6299fa-529b-4265-b351-647b0544ebbc None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:37.586421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:37.595875 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:37.622672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e072184-fecd-41f7-a49f-6628778b39af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:37.623210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e072184-fecd-41f7-a49f-6628778b39af None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.623497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e072184-fecd-41f7-a49f-6628778b39af None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.623830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e072184-fecd-41f7-a49f-6628778b39af None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.624605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e072184-fecd-41f7-a49f-6628778b39af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:37.624917 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e072184-fecd-41f7-a49f-6628778b39af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:37.626352 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e072184-fecd-41f7-a49f-6628778b39af None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:37.642478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 397/810] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:37.642478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e8df6-d838-426c-9724-404283185f80 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.642478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e8df6-d838-426c-9724-404283185f80 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.642478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e8df6-d838-426c-9724-404283185f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.658252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-9d4e8df6-d838-426c-9724-404283185f80 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:37.659568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-9d4e8df6-d838-426c-9724-404283185f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:37.691633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adab2b-f39e-4ea3-9a9c-ab6eb9a7c206 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:37.726387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e0328-4a4c-4432-8314-c30552e14cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:37.727046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e0328-4a4c-4432-8314-c30552e14cfc None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.727360 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e0328-4a4c-4432-8314-c30552e14cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.727629 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e0328-4a4c-4432-8314-c30552e14cfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.730937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 398/811] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:37.740219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e8df6-d838-426c-9724-404283185f80 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:37.815753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e0328-4a4c-4432-8314-c30552e14cfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:37.817158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e0328-4a4c-4432-8314-c30552e14cfc None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:37.836903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:37.845662 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:37.849705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e8df6-d838-426c-9724-404283185f80 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:37.886489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c6725-556b-46e2-bdb0-d9ee4dc4176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:37.887285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c6725-556b-46e2-bdb0-d9ee4dc4176c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:37.887574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c6725-556b-46e2-bdb0-d9ee4dc4176c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:37.887862 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c6725-556b-46e2-bdb0-d9ee4dc4176c None admin] PATH_INFO: `/v3/services/network` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:37.888544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c6725-556b-46e2-bdb0-d9ee4dc4176c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:37.889926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c6725-556b-46e2-bdb0-d9ee4dc4176c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-914c6725-556b-46e2-bdb0-d9ee4dc4176c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:37.902592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:37.903637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:37.905004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 399/812] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:37 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:37.905807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c6725-556b-46e2-bdb0-d9ee4dc4176c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:37.932223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 414/813] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:07:37.933297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e0328-4a4c-4432-8314-c30552e14cfc None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:38.008251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c6725-556b-46e2-bdb0-d9ee4dc4176c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:38.043102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779751d1-ffb4-4754-bf23-3d4973a8be53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:38.043910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779751d1-ffb4-4754-bf23-3d4973a8be53 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:38.044244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779751d1-ffb4-4754-bf23-3d4973a8be53 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:38.044528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779751d1-ffb4-4754-bf23-3d4973a8be53 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:38.050159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e0328-4a4c-4432-8314-c30552e14cfc None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:38.052301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779751d1-ffb4-4754-bf23-3d4973a8be53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:38.053750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779751d1-ffb4-4754-bf23-3d4973a8be53 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-779751d1-ffb4-4754-bf23-3d4973a8be53 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:38.068611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:38.069540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:38.070485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:38.070485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:38.070485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:38.070485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:38.070485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:38.070485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:38.070485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:38.070485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:38.070485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 07:07:38.070485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:38.070485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 400/814] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jun 17 07:07:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:38.070485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779751d1-ffb4-4754-bf23-3d4973a8be53 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:38.074450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:38.092735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40bfad-1f77-4ae1-bf14-444e81e0f78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:38.093958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40bfad-1f77-4ae1-bf14-444e81e0f78f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:38.094394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40bfad-1f77-4ae1-bf14-444e81e0f78f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:38.094670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40bfad-1f77-4ae1-bf14-444e81e0f78f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:38.095550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40bfad-1f77-4ae1-bf14-444e81e0f78f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:38.096739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40bfad-1f77-4ae1-bf14-444e81e0f78f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:38.101398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-be40bfad-1f77-4ae1-bf14-444e81e0f78f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:38.109340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:38.110282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:38.111131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 415/815] 10.222.0.33 () {56 vars in 1129 bytes} [Fri Jun 17 07:07:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:38.111981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40bfad-1f77-4ae1-bf14-444e81e0f78f None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:38.191840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779751d1-ffb4-4754-bf23-3d4973a8be53 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:38.221621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40bfad-1f77-4ae1-bf14-444e81e0f78f None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:38.229978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409ac30-00a8-4aed-a2c3-202c83076ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:38.230869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409ac30-00a8-4aed-a2c3-202c83076ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:38.231151 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409ac30-00a8-4aed-a2c3-202c83076ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:38.231426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409ac30-00a8-4aed-a2c3-202c83076ab2 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:38.232158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409ac30-00a8-4aed-a2c3-202c83076ab2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:38.232578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409ac30-00a8-4aed-a2c3-202c83076ab2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:38.233831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409ac30-00a8-4aed-a2c3-202c83076ab2 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:38.238410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 401/816] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:07:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:38.238410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409ac30-00a8-4aed-a2c3-202c83076ab2 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:38.262111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09b2fd-1b4f-46f4-bfc5-032db8f85756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:38.262901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09b2fd-1b4f-46f4-bfc5-032db8f85756 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:38.263285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09b2fd-1b4f-46f4-bfc5-032db8f85756 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:38.263572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09b2fd-1b4f-46f4-bfc5-032db8f85756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:38.274863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09b2fd-1b4f-46f4-bfc5-032db8f85756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:38.276822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09b2fd-1b4f-46f4-bfc5-032db8f85756 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-ad09b2fd-1b4f-46f4-bfc5-032db8f85756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:38.294716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:38.295606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:38.296535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:38.296535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:38.296535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:38.296535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:38.296535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 416/817] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:38.297396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09b2fd-1b4f-46f4-bfc5-032db8f85756 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:38.330088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:38.335445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:38.345922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409ac30-00a8-4aed-a2c3-202c83076ab2 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:38.379228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666a77d-5793-4cb5-8afd-b540f0d215fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:38.379871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666a77d-5793-4cb5-8afd-b540f0d215fe None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:38.380181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666a77d-5793-4cb5-8afd-b540f0d215fe None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:38.380461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666a77d-5793-4cb5-8afd-b540f0d215fe None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:38.381293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666a77d-5793-4cb5-8afd-b540f0d215fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:38.381605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666a77d-5793-4cb5-8afd-b540f0d215fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:38.383041 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666a77d-5793-4cb5-8afd-b540f0d215fe None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:38.405360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 402/818] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jun 17 07:07:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:38.406179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666a77d-5793-4cb5-8afd-b540f0d215fe None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:38.412944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09b2fd-1b4f-46f4-bfc5-032db8f85756 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:38.449479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6538b-cb4d-4a83-8a48-463c9bdfdeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:38.450477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6538b-cb4d-4a83-8a48-463c9bdfdeae None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:38.451051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6538b-cb4d-4a83-8a48-463c9bdfdeae None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:38.451473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6538b-cb4d-4a83-8a48-463c9bdfdeae None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:38.452390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6538b-cb4d-4a83-8a48-463c9bdfdeae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:38.452946 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6538b-cb4d-4a83-8a48-463c9bdfdeae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:38.454629 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6538b-cb4d-4a83-8a48-463c9bdfdeae None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:38.459480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 417/819] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jun 17 07:07:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:38.460493 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6538b-cb4d-4a83-8a48-463c9bdfdeae None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:38.512435 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666a77d-5793-4cb5-8afd-b540f0d215fe None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:38.548765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37431aaf-1672-4715-8ec6-a6d41e0c85c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:38.549442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37431aaf-1672-4715-8ec6-a6d41e0c85c3 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:38.549730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37431aaf-1672-4715-8ec6-a6d41e0c85c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:38.550009 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37431aaf-1672-4715-8ec6-a6d41e0c85c3 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:38.550712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37431aaf-1672-4715-8ec6-a6d41e0c85c3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:38.551026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37431aaf-1672-4715-8ec6-a6d41e0c85c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:38.552408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37431aaf-1672-4715-8ec6-a6d41e0c85c3 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:38.557746 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 403/820] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:07:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:38.558519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37431aaf-1672-4715-8ec6-a6d41e0c85c3 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:38.570598 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:38.572564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:38.577163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6538b-cb4d-4a83-8a48-463c9bdfdeae None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:38.611501 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65b5a4-272c-48dd-b129-803b3480f3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:38.612191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65b5a4-272c-48dd-b129-803b3480f3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:38.612441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65b5a4-272c-48dd-b129-803b3480f3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:38.612657 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65b5a4-272c-48dd-b129-803b3480f3a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:38.613374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65b5a4-272c-48dd-b129-803b3480f3a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:38.613690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65b5a4-272c-48dd-b129-803b3480f3a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:38.615031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65b5a4-272c-48dd-b129-803b3480f3a6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:38.632343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 418/821] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:38.632343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65b5a4-272c-48dd-b129-803b3480f3a6 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:38.672194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37431aaf-1672-4715-8ec6-a6d41e0c85c3 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:38.706655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3fb4e-b1ec-4772-9b49-f6d0bf22b892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:38.707441 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3fb4e-b1ec-4772-9b49-f6d0bf22b892 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:38.707734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3fb4e-b1ec-4772-9b49-f6d0bf22b892 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:38.708009 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3fb4e-b1ec-4772-9b49-f6d0bf22b892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:38.718169 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3fb4e-b1ec-4772-9b49-f6d0bf22b892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:38.719963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3fb4e-b1ec-4772-9b49-f6d0bf22b892 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:38.737176 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65b5a4-272c-48dd-b129-803b3480f3a6 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-89d3fb4e-b1ec-4772-9b49-f6d0bf22b892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:38.737500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:38.738522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:38.739688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:38.739688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:38.739688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:38.739688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:38.743082 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 404/822] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:38.743833 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3fb4e-b1ec-4772-9b49-f6d0bf22b892 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:38.778453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767d014-1a61-4711-b636-00c3fe1174ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:38.779317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767d014-1a61-4711-b636-00c3fe1174ba None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:38.779765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767d014-1a61-4711-b636-00c3fe1174ba None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:38.780220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767d014-1a61-4711-b636-00c3fe1174ba None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:38.781078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767d014-1a61-4711-b636-00c3fe1174ba None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:38.781526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767d014-1a61-4711-b636-00c3fe1174ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:38.783033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767d014-1a61-4711-b636-00c3fe1174ba None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:38.788566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 419/823] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jun 17 07:07:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:38.789323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767d014-1a61-4711-b636-00c3fe1174ba None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:38.809088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:38.818697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:38.853990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3fb4e-b1ec-4772-9b49-f6d0bf22b892 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:38.888359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074dc543-dc1b-475e-a62c-07f450cbc2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:38.888941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dc543-dc1b-475e-a62c-07f450cbc2ee None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:38.889232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dc543-dc1b-475e-a62c-07f450cbc2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:38.889510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dc543-dc1b-475e-a62c-07f450cbc2ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:38.890234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dc543-dc1b-475e-a62c-07f450cbc2ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:38.891604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dc543-dc1b-475e-a62c-07f450cbc2ee None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:38.898967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767d014-1a61-4711-b636-00c3fe1174ba None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:38.924107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 405/824] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jun 17 07:07:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:38.924107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074dc543-dc1b-475e-a62c-07f450cbc2ee None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:38.963233 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bac244-3e82-41bc-9ae2-cbdf20db79bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:38.964412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac244-3e82-41bc-9ae2-cbdf20db79bf None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:38.964771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac244-3e82-41bc-9ae2-cbdf20db79bf None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:38.965118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac244-3e82-41bc-9ae2-cbdf20db79bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:38.973894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bac244-3e82-41bc-9ae2-cbdf20db79bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:38.975597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bac244-3e82-41bc-9ae2-cbdf20db79bf None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-19bac244-3e82-41bc-9ae2-cbdf20db79bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:38.992565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:38.993555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:38.994453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:38.994453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:38.994453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:38.994453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:38.994453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:38.994453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:38.994453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:38.994453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:38.999369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 420/825] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:39.000362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bac244-3e82-41bc-9ae2-cbdf20db79bf None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:39.074969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074dc543-dc1b-475e-a62c-07f450cbc2ee None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:39.097878 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:39.101214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:39.125749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540cc22-e851-4218-8c7c-63e8759ae086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:39.126527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540cc22-e851-4218-8c7c-63e8759ae086 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:39.126880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540cc22-e851-4218-8c7c-63e8759ae086 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:39.127226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540cc22-e851-4218-8c7c-63e8759ae086 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:39.127856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bac244-3e82-41bc-9ae2-cbdf20db79bf None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:39.128084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540cc22-e851-4218-8c7c-63e8759ae086 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:39.128460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540cc22-e851-4218-8c7c-63e8759ae086 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:39.129912 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540cc22-e851-4218-8c7c-63e8759ae086 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:39.144593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 406/826] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:39.147629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540cc22-e851-4218-8c7c-63e8759ae086 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:39.169879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcab68-5fe3-4a31-b715-88b23a6ea7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:39.170835 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcab68-5fe3-4a31-b715-88b23a6ea7f6 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:39.171124 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcab68-5fe3-4a31-b715-88b23a6ea7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:39.171399 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcab68-5fe3-4a31-b715-88b23a6ea7f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:39.172323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fcab68-5fe3-4a31-b715-88b23a6ea7f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:39.173533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fcab68-5fe3-4a31-b715-88b23a6ea7f6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:39.204997 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 421/827] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jun 17 07:07:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:39.205972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcab68-5fe3-4a31-b715-88b23a6ea7f6 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:39.285295 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540cc22-e851-4218-8c7c-63e8759ae086 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:39.341759 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcab68-5fe3-4a31-b715-88b23a6ea7f6 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:39.346335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76dc10-23b0-4fde-8271-c9b45961f28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:39.347197 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76dc10-23b0-4fde-8271-c9b45961f28e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:39.347618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76dc10-23b0-4fde-8271-c9b45961f28e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:39.347989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76dc10-23b0-4fde-8271-c9b45961f28e None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:39.348958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76dc10-23b0-4fde-8271-c9b45961f28e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:39.349417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76dc10-23b0-4fde-8271-c9b45961f28e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:39.351749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76dc10-23b0-4fde-8271-c9b45961f28e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:39.370283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 407/828] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:39.376000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76dc10-23b0-4fde-8271-c9b45961f28e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:39.380552 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:39.386291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984ca41-59b8-40f0-abd9-8d0f5b38d5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:39.388015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984ca41-59b8-40f0-abd9-8d0f5b38d5cc None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:39.388529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984ca41-59b8-40f0-abd9-8d0f5b38d5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:39.389013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984ca41-59b8-40f0-abd9-8d0f5b38d5cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:39.396667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984ca41-59b8-40f0-abd9-8d0f5b38d5cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:39.398573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984ca41-59b8-40f0-abd9-8d0f5b38d5cc None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:39.412596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-7984ca41-59b8-40f0-abd9-8d0f5b38d5cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:39.415724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:39.417377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:39.418268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:39.418268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:39.418268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:39.418268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:39.418268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:39.418268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:39.418268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:39.418268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:39.420143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 422/829] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:39.423459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984ca41-59b8-40f0-abd9-8d0f5b38d5cc None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:39.499604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76dc10-23b0-4fde-8271-c9b45961f28e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:39.536913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4016082-2220-4bc8-9e24-87b53b311440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:39.537649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4016082-2220-4bc8-9e24-87b53b311440 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:39.537939 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4016082-2220-4bc8-9e24-87b53b311440 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:39.538218 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4016082-2220-4bc8-9e24-87b53b311440 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:39.539081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4016082-2220-4bc8-9e24-87b53b311440 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=8d3baa16824f40f3a4c783e210b262f4) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:39.539478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4016082-2220-4bc8-9e24-87b53b311440 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:39.541703 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4016082-2220-4bc8-9e24-87b53b311440 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:39.549445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 408/830] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:39 2022] GET /identity/v3/role_assignments?user.id=8d3baa16824f40f3a4c783e210b262f4&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:39.558675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984ca41-59b8-40f0-abd9-8d0f5b38d5cc None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:39.598667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5bcca-6d28-4801-a8d0-634a04f8fa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:39.599542 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813228ae-1e75-40fa-b872-9094bd06a00a None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:39.599842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813228ae-1e75-40fa-b872-9094bd06a00a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:39.600100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5bcca-6d28-4801-a8d0-634a04f8fa4f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:39.600224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813228ae-1e75-40fa-b872-9094bd06a00a None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:39.600680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5bcca-6d28-4801-a8d0-634a04f8fa4f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:39.601139 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5bcca-6d28-4801-a8d0-634a04f8fa4f None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:39.601893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 409/831] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:07:39.602427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5bcca-6d28-4801-a8d0-634a04f8fa4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:39.602932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5bcca-6d28-4801-a8d0-634a04f8fa4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:39.604858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5bcca-6d28-4801-a8d0-634a04f8fa4f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:39.607817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e83b2-7ee1-44a7-ae4d-7bad90574f3a None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:39.608139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e83b2-7ee1-44a7-ae4d-7bad90574f3a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:39.608426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e83b2-7ee1-44a7-ae4d-7bad90574f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:39.622655 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 423/832] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:39.628334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-912e83b2-7ee1-44a7-ae4d-7bad90574f3a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:39.629228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5bcca-6d28-4801-a8d0-634a04f8fa4f None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:39.629778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-912e83b2-7ee1-44a7-ae4d-7bad90574f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:39.692602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:39.712225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 410/833] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:39.733291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e83b2-7ee1-44a7-ae4d-7bad90574f3a None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:39.749593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5bcca-6d28-4801-a8d0-634a04f8fa4f None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:39.761611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:39.790485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b2c87-7f87-4fdd-98ba-8b27a7abb8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:39.791622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b2c87-7f87-4fdd-98ba-8b27a7abb8e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:39.791902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b2c87-7f87-4fdd-98ba-8b27a7abb8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:39.792211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b2c87-7f87-4fdd-98ba-8b27a7abb8e8 None admin] PATH_INFO: `/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/dc98526f14e84279b002c43ef44cabed/roles/2a15b4ff19204ac5916a0099ec8660c6` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:39.808806 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b2c87-7f87-4fdd-98ba-8b27a7abb8e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4ec9b09302d4c918ef638d794468ba3, user_id=dc98526f14e84279b002c43ef44cabed, role_id=2a15b4ff19204ac5916a0099ec8660c6)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:39.810245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b2c87-7f87-4fdd-98ba-8b27a7abb8e8 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:39.826138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 424/834] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:07:39 2022] PUT /identity/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/dc98526f14e84279b002c43ef44cabed/roles/2a15b4ff19204ac5916a0099ec8660c6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:07:39.851235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e83b2-7ee1-44a7-ae4d-7bad90574f3a None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:39.896867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689e60b-1716-4771-8083-76f58eed6306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:39.897694 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689e60b-1716-4771-8083-76f58eed6306 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:39.897986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689e60b-1716-4771-8083-76f58eed6306 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:39.898265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689e60b-1716-4771-8083-76f58eed6306 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:39.911876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689e60b-1716-4771-8083-76f58eed6306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-5689e60b-1716-4771-8083-76f58eed6306 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:39.920649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:39.921657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:39.922517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 411/835] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:07:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:39.928273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b2c87-7f87-4fdd-98ba-8b27a7abb8e8 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:40.044002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b2c87-7f87-4fdd-98ba-8b27a7abb8e8 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:40.046888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:40.080292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07f6ad-2fc4-4931-96bd-15b7f2144d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:40.080967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07f6ad-2fc4-4931-96bd-15b7f2144d12 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:40.081257 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07f6ad-2fc4-4931-96bd-15b7f2144d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:40.081541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07f6ad-2fc4-4931-96bd-15b7f2144d12 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:40.082290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07f6ad-2fc4-4931-96bd-15b7f2144d12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:40.082602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07f6ad-2fc4-4931-96bd-15b7f2144d12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:40.083964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07f6ad-2fc4-4931-96bd-15b7f2144d12 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:40.092996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 425/836] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:40.098173 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689e60b-1716-4771-8083-76f58eed6306 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:40.191531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:40.206891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689e60b-1716-4771-8083-76f58eed6306 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:40.241199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8924de2-811d-4c1a-82dc-b00b66469d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:40.242042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8924de2-811d-4c1a-82dc-b00b66469d7f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:40.242325 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8924de2-811d-4c1a-82dc-b00b66469d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:40.242599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8924de2-811d-4c1a-82dc-b00b66469d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:40.253785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8924de2-811d-4c1a-82dc-b00b66469d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:40.255236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8924de2-811d-4c1a-82dc-b00b66469d7f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-a8924de2-811d-4c1a-82dc-b00b66469d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:40.271963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:40.272879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:40.273786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:40.273786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:40.273786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:40.273786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:40.273786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 412/837] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:40.279266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07f6ad-2fc4-4931-96bd-15b7f2144d12 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:40.381303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07f6ad-2fc4-4931-96bd-15b7f2144d12 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:40.417232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98506bea-e9fb-4862-a356-2c1c2c838a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:40.417904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98506bea-e9fb-4862-a356-2c1c2c838a9b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:40.418191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98506bea-e9fb-4862-a356-2c1c2c838a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:40.418469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98506bea-e9fb-4862-a356-2c1c2c838a9b None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:40.419187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98506bea-e9fb-4862-a356-2c1c2c838a9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:40.419496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98506bea-e9fb-4862-a356-2c1c2c838a9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:40.420885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98506bea-e9fb-4862-a356-2c1c2c838a9b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:40.433727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 426/838] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:40.439130 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8924de2-811d-4c1a-82dc-b00b66469d7f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:40.546495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8924de2-811d-4c1a-82dc-b00b66469d7f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:40.580632 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f964b15-332f-4160-8ab5-40b314021d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:40.581432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f964b15-332f-4160-8ab5-40b314021d03 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:40.581713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f964b15-332f-4160-8ab5-40b314021d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:40.581988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f964b15-332f-4160-8ab5-40b314021d03 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:40.590207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f964b15-332f-4160-8ab5-40b314021d03 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:40.591658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f964b15-332f-4160-8ab5-40b314021d03 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:40.593484 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-7f964b15-332f-4160-8ab5-40b314021d03 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:40.606172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:40.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:40.608085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:40.608085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:40.608085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:40.608085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:40.608085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:40.608085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:40.608085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:40.608085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:40.608085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 07:07:40.608085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:40.608085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 413/839] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:07:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:40.613241 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98506bea-e9fb-4862-a356-2c1c2c838a9b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:40.640557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:40.711722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98506bea-e9fb-4862-a356-2c1c2c838a9b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:40.744974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2945f85-3fbe-40a3-88bf-5397e2e29991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:40.745637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945f85-3fbe-40a3-88bf-5397e2e29991 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:40.745922 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945f85-3fbe-40a3-88bf-5397e2e29991 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:40.746189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945f85-3fbe-40a3-88bf-5397e2e29991 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:40.747018 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2945f85-3fbe-40a3-88bf-5397e2e29991 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:40.747326 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2945f85-3fbe-40a3-88bf-5397e2e29991 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:40.748782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2945f85-3fbe-40a3-88bf-5397e2e29991 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:40.766114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 427/840] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jun 17 07:07:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:40.771756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f964b15-332f-4160-8ab5-40b314021d03 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:40.874956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f964b15-332f-4160-8ab5-40b314021d03 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:40.909618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21507f28-6824-4193-897c-963c7213eb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:40.910218 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21507f28-6824-4193-897c-963c7213eb5f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:40.910457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21507f28-6824-4193-897c-963c7213eb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:40.910733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21507f28-6824-4193-897c-963c7213eb5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:40.975826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:40.980616 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21507f28-6824-4193-897c-963c7213eb5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:40.982210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21507f28-6824-4193-897c-963c7213eb5f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:41.085048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 414/841] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:40 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:07:41.091066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2945f85-3fbe-40a3-88bf-5397e2e29991 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:41.173727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:41.191958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2945f85-3fbe-40a3-88bf-5397e2e29991 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:41.225389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ca58a-9b6d-431b-9e5f-8129812b4ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:41.226179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ca58a-9b6d-431b-9e5f-8129812b4ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:41.226458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ca58a-9b6d-431b-9e5f-8129812b4ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:41.226731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ca58a-9b6d-431b-9e5f-8129812b4ff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:41.240330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ca58a-9b6d-431b-9e5f-8129812b4ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:41.241789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ca58a-9b6d-431b-9e5f-8129812b4ff1 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-960ca58a-9b6d-431b-9e5f-8129812b4ff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:41.259308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:41.260297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:41.261178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:41.261178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:41.261178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:41.261178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:41.261178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 428/842] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:41.266157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21507f28-6824-4193-897c-963c7213eb5f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:41.362850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:41.366688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21507f28-6824-4193-897c-963c7213eb5f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:41.401405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254ec7c-28f9-4fd7-b3a6-31931bef0864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:41.402384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254ec7c-28f9-4fd7-b3a6-31931bef0864 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:41.402674 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254ec7c-28f9-4fd7-b3a6-31931bef0864 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:41.402950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254ec7c-28f9-4fd7-b3a6-31931bef0864 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:41.403668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254ec7c-28f9-4fd7-b3a6-31931bef0864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:41.403972 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254ec7c-28f9-4fd7-b3a6-31931bef0864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:41.405392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254ec7c-28f9-4fd7-b3a6-31931bef0864 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:41.417236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 415/843] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:41.422610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ca58a-9b6d-431b-9e5f-8129812b4ff1 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:41.528202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ca58a-9b6d-431b-9e5f-8129812b4ff1 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:41.534565 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb5025-078b-4868-9db4-1bf9406bf65f None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:41.534859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb5025-078b-4868-9db4-1bf9406bf65f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:41.535133 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb5025-078b-4868-9db4-1bf9406bf65f None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:41.537725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 416/844] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:41.542165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d099c6f-c1bb-420e-a96a-5a4f99c6f064 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:41.542462 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d099c6f-c1bb-420e-a96a-5a4f99c6f064 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:41.542745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d099c6f-c1bb-420e-a96a-5a4f99c6f064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:41.559869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-7d099c6f-c1bb-420e-a96a-5a4f99c6f064 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:41.561130 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7d099c6f-c1bb-420e-a96a-5a4f99c6f064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:41.564285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3b35f-eec0-45ad-bec3-4b3dc1c66175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:41.565113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3b35f-eec0-45ad-bec3-4b3dc1c66175 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:41.565395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3b35f-eec0-45ad-bec3-4b3dc1c66175 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:41.565668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3b35f-eec0-45ad-bec3-4b3dc1c66175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:41.572718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:41.574199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3b35f-eec0-45ad-bec3-4b3dc1c66175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:41.576612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3b35f-eec0-45ad-bec3-4b3dc1c66175 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-0df3b35f-eec0-45ad-bec3-4b3dc1c66175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:41.596549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:41.597530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:41.598426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:41.598426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:41.598426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:41.598426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:41.598426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:41.598426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:41.598426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:41.598426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:41.601080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 429/845] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:41.607450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3b35f-eec0-45ad-bec3-4b3dc1c66175 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:41.639251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 417/846] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:41.662843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d099c6f-c1bb-420e-a96a-5a4f99c6f064 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:41.728641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3b35f-eec0-45ad-bec3-4b3dc1c66175 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:41.767300 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:41.779686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1215ee-7ccb-4dbe-8a44-d6decf2f0f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:41.780400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1215ee-7ccb-4dbe-8a44-d6decf2f0f59 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:41.780757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1215ee-7ccb-4dbe-8a44-d6decf2f0f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:41.781109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1215ee-7ccb-4dbe-8a44-d6decf2f0f59 None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:41.781977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1215ee-7ccb-4dbe-8a44-d6decf2f0f59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:41.782257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d099c6f-c1bb-420e-a96a-5a4f99c6f064 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:41.782364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1215ee-7ccb-4dbe-8a44-d6decf2f0f59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:41.783882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1215ee-7ccb-4dbe-8a44-d6decf2f0f59 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:41.805971 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 430/847] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:41.809326 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1215ee-7ccb-4dbe-8a44-d6decf2f0f59 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:41.824717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf5c6b-a57e-4c79-aa84-8a733784d537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:41.826048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf5c6b-a57e-4c79-aa84-8a733784d537 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:41.826579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf5c6b-a57e-4c79-aa84-8a733784d537 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:41.827054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf5c6b-a57e-4c79-aa84-8a733784d537 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:41.827956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf5c6b-a57e-4c79-aa84-8a733784d537 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:41.829634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf5c6b-a57e-4c79-aa84-8a733784d537 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-c6bf5c6b-a57e-4c79-aa84-8a733784d537 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:41.843437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:41.844547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 07:07:41.845372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:41.847894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 418/848] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:07:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:41.857436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf5c6b-a57e-4c79-aa84-8a733784d537 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:41.883136 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:41.929649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1215ee-7ccb-4dbe-8a44-d6decf2f0f59 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:41.963066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e39ae7-ae6a-4354-b6dc-b3a42f8f7f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:41.963730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e39ae7-ae6a-4354-b6dc-b3a42f8f7f3f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:41.964015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e39ae7-ae6a-4354-b6dc-b3a42f8f7f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:41.964345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e39ae7-ae6a-4354-b6dc-b3a42f8f7f3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:41.965177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e39ae7-ae6a-4354-b6dc-b3a42f8f7f3f None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=004e5909f9834cfea890063bb0083d7c) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:41.965490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e39ae7-ae6a-4354-b6dc-b3a42f8f7f3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:41.966916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e39ae7-ae6a-4354-b6dc-b3a42f8f7f3f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:41.970498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf5c6b-a57e-4c79-aa84-8a733784d537 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:41.972691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 431/849] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:41 2022] GET /identity/v3/role_assignments?user.id=004e5909f9834cfea890063bb0083d7c&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:41.973987 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc02424-a359-42a7-adf6-3d35a95624c6 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:41.974274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc02424-a359-42a7-adf6-3d35a95624c6 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:41.974554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc02424-a359-42a7-adf6-3d35a95624c6 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:41.975925 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 432/850] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:41.982361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267133cb-6f9f-4d11-a711-dc7c7b54dd55 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:41.982752 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267133cb-6f9f-4d11-a711-dc7c7b54dd55 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:41.983138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267133cb-6f9f-4d11-a711-dc7c7b54dd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:42.001053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-267133cb-6f9f-4d11-a711-dc7c7b54dd55 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:42.002484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-267133cb-6f9f-4d11-a711-dc7c7b54dd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:42.022709 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09716138-3d51-4dd3-a29a-f536fa372cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:42.023702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09716138-3d51-4dd3-a29a-f536fa372cbc None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:42.024269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09716138-3d51-4dd3-a29a-f536fa372cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:42.024714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09716138-3d51-4dd3-a29a-f536fa372cbc None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:42.026149 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09716138-3d51-4dd3-a29a-f536fa372cbc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:42.026621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09716138-3d51-4dd3-a29a-f536fa372cbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:42.028344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09716138-3d51-4dd3-a29a-f536fa372cbc None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:42.038969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 419/851] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:07:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:42.040722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09716138-3d51-4dd3-a29a-f536fa372cbc None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:42.061913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:42.089780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 433/852] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:42.108633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267133cb-6f9f-4d11-a711-dc7c7b54dd55 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:42.160909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09716138-3d51-4dd3-a29a-f536fa372cbc None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:42.164584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:42.201566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b7a21-9ea0-40dd-8b37-7ae06ff7f829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:42.202238 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b7a21-9ea0-40dd-8b37-7ae06ff7f829 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:42.202667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b7a21-9ea0-40dd-8b37-7ae06ff7f829 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:42.202949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b7a21-9ea0-40dd-8b37-7ae06ff7f829 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:42.204029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b7a21-9ea0-40dd-8b37-7ae06ff7f829 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:42.204440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b7a21-9ea0-40dd-8b37-7ae06ff7f829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:42.205815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b7a21-9ea0-40dd-8b37-7ae06ff7f829 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:42.210860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 420/853] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:07:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:42.211876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510afe6d-afdc-4bc9-8e09-2d859385728b None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:42.216307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510afe6d-afdc-4bc9-8e09-2d859385728b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:42.216307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510afe6d-afdc-4bc9-8e09-2d859385728b None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:42.216307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 421/854] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:42.220995 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f910e0e-19c9-45e7-b782-a7b284621a4d None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:42.221791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f910e0e-19c9-45e7-b782-a7b284621a4d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:42.222217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f910e0e-19c9-45e7-b782-a7b284621a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:42.230697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267133cb-6f9f-4d11-a711-dc7c7b54dd55 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:42.245320 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-5f910e0e-19c9-45e7-b782-a7b284621a4d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:42.247681 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5f910e0e-19c9-45e7-b782-a7b284621a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:42.276594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58390fe-ed81-4a17-a191-d9981b95e568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:42.277584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58390fe-ed81-4a17-a191-d9981b95e568 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:42.277872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58390fe-ed81-4a17-a191-d9981b95e568 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:42.278214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58390fe-ed81-4a17-a191-d9981b95e568 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:42.278894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58390fe-ed81-4a17-a191-d9981b95e568 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:42.280421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58390fe-ed81-4a17-a191-d9981b95e568 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-e58390fe-ed81-4a17-a191-d9981b95e568 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:42.295095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:42.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:42.297320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 434/855] 10.222.0.33 () {56 vars in 1132 bytes} [Fri Jun 17 07:07:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:42.303864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58390fe-ed81-4a17-a191-d9981b95e568 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:42.341226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 422/856] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:42.363743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f910e0e-19c9-45e7-b782-a7b284621a4d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:42.385908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:42.445694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58390fe-ed81-4a17-a191-d9981b95e568 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:42.459978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:42.483613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d437c4-7efe-4419-aaf0-ecda5a03801e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:42.484245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d437c4-7efe-4419-aaf0-ecda5a03801e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:42.484533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d437c4-7efe-4419-aaf0-ecda5a03801e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:42.484812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d437c4-7efe-4419-aaf0-ecda5a03801e None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:42.485533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d437c4-7efe-4419-aaf0-ecda5a03801e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:42.485821 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d437c4-7efe-4419-aaf0-ecda5a03801e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:42.487263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d437c4-7efe-4419-aaf0-ecda5a03801e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:42.491983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f910e0e-19c9-45e7-b782-a7b284621a4d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:42.497059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 435/857] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jun 17 07:07:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:42.498368 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d437c4-7efe-4419-aaf0-ecda5a03801e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:42.539442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ddc41-e2d3-43f5-8430-29f5bd0d7863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:42.540685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ddc41-e2d3-43f5-8430-29f5bd0d7863 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:42.541197 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ddc41-e2d3-43f5-8430-29f5bd0d7863 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:42.541678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ddc41-e2d3-43f5-8430-29f5bd0d7863 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:42.554984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ddc41-e2d3-43f5-8430-29f5bd0d7863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-ea2ddc41-e2d3-43f5-8430-29f5bd0d7863 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:42.562968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:42.563961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:42.564891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 423/858] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:07:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:42.569755 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ddc41-e2d3-43f5-8430-29f5bd0d7863 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:42.621904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d437c4-7efe-4419-aaf0-ecda5a03801e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:42.661552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:42.677362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d06a9-419d-42d2-905f-5395e9be2503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:42.678346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d06a9-419d-42d2-905f-5395e9be2503 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:42.678784 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d06a9-419d-42d2-905f-5395e9be2503 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:42.679209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d06a9-419d-42d2-905f-5395e9be2503 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:42.680135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d06a9-419d-42d2-905f-5395e9be2503 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:42.680597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d06a9-419d-42d2-905f-5395e9be2503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:42.682191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d06a9-419d-42d2-905f-5395e9be2503 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:42.688200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 436/859] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jun 17 07:07:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:42.824166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ddc41-e2d3-43f5-8430-29f5bd0d7863 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:42.868198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b976a-6dcf-4abf-9a1b-9bfcdac502df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:42.868910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b976a-6dcf-4abf-9a1b-9bfcdac502df None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:42.869194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b976a-6dcf-4abf-9a1b-9bfcdac502df None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:42.869471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b976a-6dcf-4abf-9a1b-9bfcdac502df None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:42.870219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b976a-6dcf-4abf-9a1b-9bfcdac502df None admin] RBAC: Adding query filter params (name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:42.870526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b976a-6dcf-4abf-9a1b-9bfcdac502df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:42.871901 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b976a-6dcf-4abf-9a1b-9bfcdac502df None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:42.880544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 424/860] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:07:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:42.885717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d06a9-419d-42d2-905f-5395e9be2503 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:42.937031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:42.998631 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d06a9-419d-42d2-905f-5395e9be2503 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:43.034526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e809b-8686-473d-b5fa-aac1c64e1f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:43.035327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e809b-8686-473d-b5fa-aac1c64e1f89 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:43.035616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e809b-8686-473d-b5fa-aac1c64e1f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:43.035897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e809b-8686-473d-b5fa-aac1c64e1f89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:43.047033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e809b-8686-473d-b5fa-aac1c64e1f89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:43.048534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e809b-8686-473d-b5fa-aac1c64e1f89 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-5d7e809b-8686-473d-b5fa-aac1c64e1f89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:43.066083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:43.067033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:43.067928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:43.067928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:43.067928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:43.067928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:43.067928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 437/861] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:43.073527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b976a-6dcf-4abf-9a1b-9bfcdac502df None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:43.177341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b976a-6dcf-4abf-9a1b-9bfcdac502df None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:43.208317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:43.215813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10164f-cea1-430c-acfe-d8dcff76151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:43.216513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10164f-cea1-430c-acfe-d8dcff76151a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:43.216802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10164f-cea1-430c-acfe-d8dcff76151a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:43.217082 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10164f-cea1-430c-acfe-d8dcff76151a None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:43.217805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10164f-cea1-430c-acfe-d8dcff76151a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:43.218118 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10164f-cea1-430c-acfe-d8dcff76151a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:43.219484 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10164f-cea1-430c-acfe-d8dcff76151a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:43.231558 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 425/862] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:43.236642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e809b-8686-473d-b5fa-aac1c64e1f89 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:43.335054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:43.349816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e809b-8686-473d-b5fa-aac1c64e1f89 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:43.387717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5d447-8a90-4da8-96d2-6147e190350a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:43.388997 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5d447-8a90-4da8-96d2-6147e190350a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:43.389290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5d447-8a90-4da8-96d2-6147e190350a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:43.389569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5d447-8a90-4da8-96d2-6147e190350a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:43.398255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5d447-8a90-4da8-96d2-6147e190350a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:43.399875 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5d447-8a90-4da8-96d2-6147e190350a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-d6e5d447-8a90-4da8-96d2-6147e190350a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:43.416464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:43.417411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:43.418339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:43.418339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:43.418339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:43.418339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:43.418339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:43.418339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:43.418339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:43.418339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:43.418339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 07:07:43.418339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:43.422328 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 438/863] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jun 17 07:07:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:43.423669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10164f-cea1-430c-acfe-d8dcff76151a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:43.528896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10164f-cea1-430c-acfe-d8dcff76151a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:43.565896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a089-2bd3-42ef-bf06-b0b6d8c0e176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:43.566912 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a089-2bd3-42ef-bf06-b0b6d8c0e176 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:43.567395 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a089-2bd3-42ef-bf06-b0b6d8c0e176 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:43.567868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a089-2bd3-42ef-bf06-b0b6d8c0e176 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:43.568921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950a089-2bd3-42ef-bf06-b0b6d8c0e176 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:43.569450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950a089-2bd3-42ef-bf06-b0b6d8c0e176 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:43.571121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950a089-2bd3-42ef-bf06-b0b6d8c0e176 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:43.592397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 426/864] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jun 17 07:07:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:43.594865 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5d447-8a90-4da8-96d2-6147e190350a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:43.696298 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5d447-8a90-4da8-96d2-6147e190350a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:43.729846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c4594-edd8-4ab7-b16f-fd0273fa9ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:43.730828 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c4594-edd8-4ab7-b16f-fd0273fa9ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:43.731314 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c4594-edd8-4ab7-b16f-fd0273fa9ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:43.731789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c4594-edd8-4ab7-b16f-fd0273fa9ec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:43.756595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:43.801609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c4594-edd8-4ab7-b16f-fd0273fa9ec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:43.803409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c4594-edd8-4ab7-b16f-fd0273fa9ec5 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:43.906552 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 439/865] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:07:43.912097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a089-2bd3-42ef-bf06-b0b6d8c0e176 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:43.964462 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:44.016089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a089-2bd3-42ef-bf06-b0b6d8c0e176 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:44.051271 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5ccb7-0608-4d3b-9a48-3342de2ecd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:44.052494 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5ccb7-0608-4d3b-9a48-3342de2ecd7f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:44.052982 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5ccb7-0608-4d3b-9a48-3342de2ecd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:44.053456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5ccb7-0608-4d3b-9a48-3342de2ecd7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:44.063925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5ccb7-0608-4d3b-9a48-3342de2ecd7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:44.065633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5ccb7-0608-4d3b-9a48-3342de2ecd7f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-ebc5ccb7-0608-4d3b-9a48-3342de2ecd7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:44.083890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:44.085261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:44.086151 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:44.086151 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:44.086151 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:44.086151 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:44.088464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 427/866] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:44.094083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c4594-edd8-4ab7-b16f-fd0273fa9ec5 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:44.149192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:44.195350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c4594-edd8-4ab7-b16f-fd0273fa9ec5 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:44.232113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8fd9b-0141-4ad3-87c3-4edc1fafee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:44.233192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8fd9b-0141-4ad3-87c3-4edc1fafee98 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:44.233702 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8fd9b-0141-4ad3-87c3-4edc1fafee98 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:44.234187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8fd9b-0141-4ad3-87c3-4edc1fafee98 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:44.235158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8fd9b-0141-4ad3-87c3-4edc1fafee98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:44.235683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8fd9b-0141-4ad3-87c3-4edc1fafee98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:44.237368 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8fd9b-0141-4ad3-87c3-4edc1fafee98 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:44.251325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 440/867] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:44.257046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5ccb7-0608-4d3b-9a48-3342de2ecd7f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:44.290462 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91494d-fa7b-41be-8441-6dd1c4068767 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:44.291163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91494d-fa7b-41be-8441-6dd1c4068767 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:44.291646 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91494d-fa7b-41be-8441-6dd1c4068767 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:44.293987 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 441/868] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:44.300973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa61dd5-341d-4138-b87c-95254114da0d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:44.301503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa61dd5-341d-4138-b87c-95254114da0d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:44.301987 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa61dd5-341d-4138-b87c-95254114da0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:44.319353 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3aa61dd5-341d-4138-b87c-95254114da0d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:44.320888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3aa61dd5-341d-4138-b87c-95254114da0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:44.374004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5ccb7-0608-4d3b-9a48-3342de2ecd7f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:44.376191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:44.406593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 442/869] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:44.419581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f755e3-6471-4269-9bb2-12e599abcb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:44.420707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f755e3-6471-4269-9bb2-12e599abcb87 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:44.421221 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f755e3-6471-4269-9bb2-12e599abcb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:44.421705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f755e3-6471-4269-9bb2-12e599abcb87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:44.429364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa61dd5-341d-4138-b87c-95254114da0d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:44.433090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f755e3-6471-4269-9bb2-12e599abcb87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:44.434562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f755e3-6471-4269-9bb2-12e599abcb87 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-64f755e3-6471-4269-9bb2-12e599abcb87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:44.449927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:44.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:44.451792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:44.451792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:44.451792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:44.451792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:44.451792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:44.451792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:44.451792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:44.451792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:44.455789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 428/870] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:44.457179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dc7da-dbb7-4665-bb23-8b313413f81d None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:44.457468 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dc7da-dbb7-4665-bb23-8b313413f81d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:44.457736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dc7da-dbb7-4665-bb23-8b313413f81d None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:44.459159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 429/871] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:44.459693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3dc7da-dbb7-4665-bb23-8b313413f81d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:44.476919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:44.536574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa61dd5-341d-4138-b87c-95254114da0d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:44.566044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3dc7da-dbb7-4665-bb23-8b313413f81d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:44.574140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae5824-6a0e-401f-9df7-1b053320e48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:44.575219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae5824-6a0e-401f-9df7-1b053320e48c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:44.575715 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae5824-6a0e-401f-9df7-1b053320e48c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:44.576059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae5824-6a0e-401f-9df7-1b053320e48c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:44.576739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae5824-6a0e-401f-9df7-1b053320e48c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:44.578091 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae5824-6a0e-401f-9df7-1b053320e48c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-e9ae5824-6a0e-401f-9df7-1b053320e48c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:44.590694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:44.591709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:44.592666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 443/872] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jun 17 07:07:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:44.593483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff27f8a-7f53-43ce-888d-f37b0ec5e6be None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:44.593483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff27f8a-7f53-43ce-888d-f37b0ec5e6be None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:44.594134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff27f8a-7f53-43ce-888d-f37b0ec5e6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:44.606787 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d6051-963e-4c2c-bb5b-adfe190a0a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:44.607449 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d6051-963e-4c2c-bb5b-adfe190a0a0f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:44.607732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d6051-963e-4c2c-bb5b-adfe190a0a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:44.608012 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d6051-963e-4c2c-bb5b-adfe190a0a0f None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:44.608822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d6051-963e-4c2c-bb5b-adfe190a0a0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:44.609134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d6051-963e-4c2c-bb5b-adfe190a0a0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:44.609921 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-5ff27f8a-7f53-43ce-888d-f37b0ec5e6be None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:44.610547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d6051-963e-4c2c-bb5b-adfe190a0a0f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:44.611102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-5ff27f8a-7f53-43ce-888d-f37b0ec5e6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:44.628570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 430/873] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:44.628570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d6051-963e-4c2c-bb5b-adfe190a0a0f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:44.654347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:44.692081 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 444/874] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:44.692081 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff27f8a-7f53-43ce-888d-f37b0ec5e6be None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:44.724460 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:44.747477 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d6051-963e-4c2c-bb5b-adfe190a0a0f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:44.781815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828d54b-e61e-4535-b41c-8281415d070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:44.782484 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828d54b-e61e-4535-b41c-8281415d070a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:44.782774 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828d54b-e61e-4535-b41c-8281415d070a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:44.783060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828d54b-e61e-4535-b41c-8281415d070a None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:44.783763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828d54b-e61e-4535-b41c-8281415d070a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:44.784110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828d54b-e61e-4535-b41c-8281415d070a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:44.785475 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828d54b-e61e-4535-b41c-8281415d070a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:44.790240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 431/875] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jun 17 07:07:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:44.791033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828d54b-e61e-4535-b41c-8281415d070a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:44.809826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff27f8a-7f53-43ce-888d-f37b0ec5e6be None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:44.852192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03ecae-6f79-4853-9a98-dcfb72e0e2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:44.852909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03ecae-6f79-4853-9a98-dcfb72e0e2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:44.853243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03ecae-6f79-4853-9a98-dcfb72e0e2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:44.853519 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03ecae-6f79-4853-9a98-dcfb72e0e2f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:44.854396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03ecae-6f79-4853-9a98-dcfb72e0e2f4 None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=dc98526f14e84279b002c43ef44cabed) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:44.854716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03ecae-6f79-4853-9a98-dcfb72e0e2f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:44.856159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03ecae-6f79-4853-9a98-dcfb72e0e2f4 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:44.864491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 445/876] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:44 2022] GET /identity/v3/role_assignments?user.id=dc98526f14e84279b002c43ef44cabed&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:44.865026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03ecae-6f79-4853-9a98-dcfb72e0e2f4 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:44.883697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:44.905467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828d54b-e61e-4535-b41c-8281415d070a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:44.955625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c5b81-ab09-4e3a-9f8f-3db0430a37ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:44.956366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c5b81-ab09-4e3a-9f8f-3db0430a37ae None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:44.956653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c5b81-ab09-4e3a-9f8f-3db0430a37ae None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:44.956963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c5b81-ab09-4e3a-9f8f-3db0430a37ae None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:44.957717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c5b81-ab09-4e3a-9f8f-3db0430a37ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:44.959863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c5b81-ab09-4e3a-9f8f-3db0430a37ae None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:44.980807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 432/877] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:07:44 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:45.001709 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e8e84-ddb7-4690-bf00-1bae5795a399 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:45.002060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e8e84-ddb7-4690-bf00-1bae5795a399 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:45.002285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e8e84-ddb7-4690-bf00-1bae5795a399 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:45.003792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 433/878] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:07:45.010012 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61246590-5254-4dfc-a192-33cff9139feb None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:45.010338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61246590-5254-4dfc-a192-33cff9139feb None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:45.010590 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61246590-5254-4dfc-a192-33cff9139feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:45.016356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:45.020776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03ecae-6f79-4853-9a98-dcfb72e0e2f4 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:45.032145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-61246590-5254-4dfc-a192-33cff9139feb None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:45.033560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-61246590-5254-4dfc-a192-33cff9139feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:45.080728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d956aa-88ee-4d52-9249-d1a6e45c27a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:45.081426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d956aa-88ee-4d52-9249-d1a6e45c27a4 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:45.081712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d956aa-88ee-4d52-9249-d1a6e45c27a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:45.081990 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d956aa-88ee-4d52-9249-d1a6e45c27a4 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:45.082712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d956aa-88ee-4d52-9249-d1a6e45c27a4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:45.083026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d956aa-88ee-4d52-9249-d1a6e45c27a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:45.084463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d956aa-88ee-4d52-9249-d1a6e45c27a4 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:45.090617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 446/879] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jun 17 07:07:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:45.139727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 434/880] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:45.166985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d956aa-88ee-4d52-9249-d1a6e45c27a4 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:45.283189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d956aa-88ee-4d52-9249-d1a6e45c27a4 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:45.319052 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce5800-be71-4919-8938-9333c62aee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:45.320463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce5800-be71-4919-8938-9333c62aee37 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:45.321130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce5800-be71-4919-8938-9333c62aee37 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:45.322289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce5800-be71-4919-8938-9333c62aee37 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:45.323298 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce5800-be71-4919-8938-9333c62aee37 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:45.325560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce5800-be71-4919-8938-9333c62aee37 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:45.342914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 447/881] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:07:45 2022] POST /identity/v3/services => generated 269 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:47.128997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce652d-7d74-4daa-9c4c-70902a7fcf55 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.129562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce652d-7d74-4daa-9c4c-70902a7fcf55 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.129562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce652d-7d74-4daa-9c4c-70902a7fcf55 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.131300 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 435/882] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:47.137358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2b4c9-7999-436f-83f7-1cc83d749905 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.137879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2b4c9-7999-436f-83f7-1cc83d749905 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.137981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2b4c9-7999-436f-83f7-1cc83d749905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.157504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f3b2b4c9-7999-436f-83f7-1cc83d749905 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:47.158716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f3b2b4c9-7999-436f-83f7-1cc83d749905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:47.174457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:47.208184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd8e34-d139-4d80-b083-2ccf1b180572 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.208522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd8e34-d139-4d80-b083-2ccf1b180572 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.208743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd8e34-d139-4d80-b083-2ccf1b180572 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.210402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 436/883] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:07:47.216855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d1792-36b3-4f27-b2a1-a1a1a7fa1c3c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.217159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d1792-36b3-4f27-b2a1-a1a1a7fa1c3c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.217417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d1792-36b3-4f27-b2a1-a1a1a7fa1c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.235479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-2d7d1792-36b3-4f27-b2a1-a1a1a7fa1c3c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:47.237209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-2d7d1792-36b3-4f27-b2a1-a1a1a7fa1c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:47.247258 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 448/884] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:47.270494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2b4c9-7999-436f-83f7-1cc83d749905 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:47.314678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 437/885] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:47.316962 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869f0e6-82a9-4c8e-8a86-eccc47db6193 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.317458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869f0e6-82a9-4c8e-8a86-eccc47db6193 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.318112 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869f0e6-82a9-4c8e-8a86-eccc47db6193 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.320509 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 438/886] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:47.329250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875fc3f-20b7-45aa-9da6-dde6d238940c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.329540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875fc3f-20b7-45aa-9da6-dde6d238940c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.329824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875fc3f-20b7-45aa-9da6-dde6d238940c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.344647 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:47.351540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-8875fc3f-20b7-45aa-9da6-dde6d238940c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:47.353320 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8875fc3f-20b7-45aa-9da6-dde6d238940c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:47.383626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2b4c9-7999-436f-83f7-1cc83d749905 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:47.419218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c08be6-d310-447f-b3de-d8371f0929e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:47.420055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c08be6-d310-447f-b3de-d8371f0929e7 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.420372 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c08be6-d310-447f-b3de-d8371f0929e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.420644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c08be6-d310-447f-b3de-d8371f0929e7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.433041 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c08be6-d310-447f-b3de-d8371f0929e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:47.443942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 439/887] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:47.443942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875fc3f-20b7-45aa-9da6-dde6d238940c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-b2c08be6-d310-447f-b3de-d8371f0929e7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:47.444265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:47.445159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:47.446000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 449/888] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jun 17 07:07:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:47.458197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c08be6-d310-447f-b3de-d8371f0929e7 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:47.470077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:47.542720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875fc3f-20b7-45aa-9da6-dde6d238940c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:47.561590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c08be6-d310-447f-b3de-d8371f0929e7 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:47.575061 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909b0ea-103c-4e01-a0bc-0786f9f522ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:47.575852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909b0ea-103c-4e01-a0bc-0786f9f522ca None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.576163 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909b0ea-103c-4e01-a0bc-0786f9f522ca None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.576441 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909b0ea-103c-4e01-a0bc-0786f9f522ca None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.577088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909b0ea-103c-4e01-a0bc-0786f9f522ca None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:47.578447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909b0ea-103c-4e01-a0bc-0786f9f522ca None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:47.581258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-f909b0ea-103c-4e01-a0bc-0786f9f522ca None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:47.591498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:47.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:47.593687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 440/889] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:07:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:47.594488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909b0ea-103c-4e01-a0bc-0786f9f522ca None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:47.596596 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618baccc-8e3d-4af7-9394-14e7ce336651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:47.597261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618baccc-8e3d-4af7-9394-14e7ce336651 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.597578 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618baccc-8e3d-4af7-9394-14e7ce336651 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.597878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618baccc-8e3d-4af7-9394-14e7ce336651 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.598586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618baccc-8e3d-4af7-9394-14e7ce336651 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:47.598890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618baccc-8e3d-4af7-9394-14e7ce336651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:47.600268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618baccc-8e3d-4af7-9394-14e7ce336651 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:47.608836 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 450/890] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jun 17 07:07:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:47.610028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecace9-b0c7-41a7-868f-93d2f636c356 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.610318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecace9-b0c7-41a7-868f-93d2f636c356 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.610689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecace9-b0c7-41a7-868f-93d2f636c356 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.612038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 451/891] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:47.612757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecace9-b0c7-41a7-868f-93d2f636c356 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:47.678177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:47.697386 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909b0ea-103c-4e01-a0bc-0786f9f522ca None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:47.712251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecace9-b0c7-41a7-868f-93d2f636c356 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:47.729524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1002dce-5e46-4bf4-880d-2911e3b92889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:47.730178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1002dce-5e46-4bf4-880d-2911e3b92889 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.730461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1002dce-5e46-4bf4-880d-2911e3b92889 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.730733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1002dce-5e46-4bf4-880d-2911e3b92889 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.731436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1002dce-5e46-4bf4-880d-2911e3b92889 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:47.732824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1002dce-5e46-4bf4-880d-2911e3b92889 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:47.745733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 441/892] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:07:47 2022] POST /identity/v3/services => generated 241 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:47.746728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1002dce-5e46-4bf4-880d-2911e3b92889 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:47.751453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da191ba-3f4e-44f7-96c3-550cb7d9d745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:47.752161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da191ba-3f4e-44f7-96c3-550cb7d9d745 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.752447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da191ba-3f4e-44f7-96c3-550cb7d9d745 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.752722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da191ba-3f4e-44f7-96c3-550cb7d9d745 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.753440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da191ba-3f4e-44f7-96c3-550cb7d9d745 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:47.753746 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da191ba-3f4e-44f7-96c3-550cb7d9d745 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:47.755148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da191ba-3f4e-44f7-96c3-550cb7d9d745 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:47.760874 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 452/893] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:07:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:47.762225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182b262-baad-448a-bb35-02a076f080e2 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.762515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182b262-baad-448a-bb35-02a076f080e2 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.762793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182b262-baad-448a-bb35-02a076f080e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.786689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-b182b262-baad-448a-bb35-02a076f080e2 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:47.787905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b182b262-baad-448a-bb35-02a076f080e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:47.832720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:47.876645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 453/894] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:47.877879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182b262-baad-448a-bb35-02a076f080e2 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:47.897077 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1002dce-5e46-4bf4-880d-2911e3b92889 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:47.938682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6407b0-0453-4730-8d5b-505d52f641f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:47.940089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6407b0-0453-4730-8d5b-505d52f641f6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:47.940089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6407b0-0453-4730-8d5b-505d52f641f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:47.940283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6407b0-0453-4730-8d5b-505d52f641f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:47.951851 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6407b0-0453-4730-8d5b-505d52f641f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:47.953356 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6407b0-0453-4730-8d5b-505d52f641f6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:47.955274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-fd6407b0-0453-4730-8d5b-505d52f641f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:47.971045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:47.972208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:47.977735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:47.977735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:47.977735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:47.977735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:47.977735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 442/895] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:47.977735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6407b0-0453-4730-8d5b-505d52f641f6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:48.002363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182b262-baad-448a-bb35-02a076f080e2 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:48.039214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bf413-c2a8-4d9d-a032-2f9a19d31dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:48.039881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bf413-c2a8-4d9d-a032-2f9a19d31dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:48.040196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bf413-c2a8-4d9d-a032-2f9a19d31dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:48.040547 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bf413-c2a8-4d9d-a032-2f9a19d31dd4 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:48.041256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bf413-c2a8-4d9d-a032-2f9a19d31dd4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:48.041563 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bf413-c2a8-4d9d-a032-2f9a19d31dd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:48.042934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bf413-c2a8-4d9d-a032-2f9a19d31dd4 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:48.047919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 454/896] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:07:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:48.048722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bf413-c2a8-4d9d-a032-2f9a19d31dd4 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:48.081067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6407b0-0453-4730-8d5b-505d52f641f6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:48.096614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:48.124353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc52ed4-4c42-4e23-95dd-4f38a5446bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:48.125178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc52ed4-4c42-4e23-95dd-4f38a5446bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:48.125466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc52ed4-4c42-4e23-95dd-4f38a5446bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:48.125745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc52ed4-4c42-4e23-95dd-4f38a5446bd5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:48.126399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc52ed4-4c42-4e23-95dd-4f38a5446bd5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:48.127774 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc52ed4-4c42-4e23-95dd-4f38a5446bd5 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-acc52ed4-4c42-4e23-95dd-4f38a5446bd5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:48.143666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:48.150593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:48.151609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 443/897] 10.222.0.33 () {56 vars in 1132 bytes} [Fri Jun 17 07:07:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:48.152883 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc52ed4-4c42-4e23-95dd-4f38a5446bd5 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:48.157665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bf413-c2a8-4d9d-a032-2f9a19d31dd4 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:48.183873 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:48.196096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6518a5b-b20c-4cae-9b1c-3100d63f2db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:48.196762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6518a5b-b20c-4cae-9b1c-3100d63f2db5 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:48.197050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6518a5b-b20c-4cae-9b1c-3100d63f2db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:48.197323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6518a5b-b20c-4cae-9b1c-3100d63f2db5 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:48.198041 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6518a5b-b20c-4cae-9b1c-3100d63f2db5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:48.198342 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6518a5b-b20c-4cae-9b1c-3100d63f2db5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:48.199700 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6518a5b-b20c-4cae-9b1c-3100d63f2db5 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:48.211562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 455/898] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:48.216837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6518a5b-b20c-4cae-9b1c-3100d63f2db5 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:48.257135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc52ed4-4c42-4e23-95dd-4f38a5446bd5 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:48.293477 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4189e-f4d1-4059-84c7-d5156a44b89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:48.294142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4189e-f4d1-4059-84c7-d5156a44b89d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:48.294425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4189e-f4d1-4059-84c7-d5156a44b89d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:48.294705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4189e-f4d1-4059-84c7-d5156a44b89d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:48.295486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4189e-f4d1-4059-84c7-d5156a44b89d None admin] RBAC: Adding query filter params (interface=public, service_id=b141343d02f34b44938cdd7b0f279060, region_id=RegionOne) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:48.295798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4189e-f4d1-4059-84c7-d5156a44b89d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:48.297211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4189e-f4d1-4059-84c7-d5156a44b89d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:48.307237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 444/899] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jun 17 07:07:48 2022] GET /identity/v3/endpoints?service_id=b141343d02f34b44938cdd7b0f279060&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:48.307237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4189e-f4d1-4059-84c7-d5156a44b89d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:48.320906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6518a5b-b20c-4cae-9b1c-3100d63f2db5 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:48.354178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:48.356275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bca355-b162-44a0-8ff0-d52376f5a5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:48.357019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bca355-b162-44a0-8ff0-d52376f5a5eb None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:48.357314 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bca355-b162-44a0-8ff0-d52376f5a5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:48.357594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bca355-b162-44a0-8ff0-d52376f5a5eb None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:48.358373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bca355-b162-44a0-8ff0-d52376f5a5eb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:48.358703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bca355-b162-44a0-8ff0-d52376f5a5eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:48.360050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bca355-b162-44a0-8ff0-d52376f5a5eb None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:48.365106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 456/900] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jun 17 07:07:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:48.370972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bca355-b162-44a0-8ff0-d52376f5a5eb None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:48.422902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:48.424486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4189e-f4d1-4059-84c7-d5156a44b89d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:48.461321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda785f7-4ee7-462b-92ac-97c9d5c48698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:48.461989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda785f7-4ee7-462b-92ac-97c9d5c48698 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:48.462334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda785f7-4ee7-462b-92ac-97c9d5c48698 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:48.462612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda785f7-4ee7-462b-92ac-97c9d5c48698 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:48.470309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda785f7-4ee7-462b-92ac-97c9d5c48698 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:48.471752 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda785f7-4ee7-462b-92ac-97c9d5c48698 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:48.479207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bca355-b162-44a0-8ff0-d52376f5a5eb None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-bda785f7-4ee7-462b-92ac-97c9d5c48698 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:48.487365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:48.488397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:48.489388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:48.489388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:48.489388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:48.489388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:48.489388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:48.489388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:48.489388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:48.489388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:48.489388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 07:07:48.489388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:48.489388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 445/901] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jun 17 07:07:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:48.489388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda785f7-4ee7-462b-92ac-97c9d5c48698 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:48.526858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb84ac-2ad4-4fab-9c7a-cbc02d72b150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:48.527905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb84ac-2ad4-4fab-9c7a-cbc02d72b150 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:48.528406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb84ac-2ad4-4fab-9c7a-cbc02d72b150 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:48.528884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb84ac-2ad4-4fab-9c7a-cbc02d72b150 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:48.529742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb84ac-2ad4-4fab-9c7a-cbc02d72b150 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:48.531801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb84ac-2ad4-4fab-9c7a-cbc02d72b150 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:48.541102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 457/902] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jun 17 07:07:48 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:07:48.542145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb84ac-2ad4-4fab-9c7a-cbc02d72b150 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:48.599409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:48.604171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda785f7-4ee7-462b-92ac-97c9d5c48698 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:48.643205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e94c7-5556-4dd9-a786-69e9c96c1de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:48.644076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e94c7-5556-4dd9-a786-69e9c96c1de0 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:48.644708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e94c7-5556-4dd9-a786-69e9c96c1de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:48.644828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e94c7-5556-4dd9-a786-69e9c96c1de0 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:48.645766 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e94c7-5556-4dd9-a786-69e9c96c1de0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:48.646180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e94c7-5556-4dd9-a786-69e9c96c1de0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:48.647862 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e94c7-5556-4dd9-a786-69e9c96c1de0 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:48.654718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 446/903] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jun 17 07:07:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:48.660371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e94c7-5556-4dd9-a786-69e9c96c1de0 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:48.697660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb84ac-2ad4-4fab-9c7a-cbc02d72b150 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:48.708968 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:48.745155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27788f-ee9e-45ec-9325-69dcb7821f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:48.745762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27788f-ee9e-45ec-9325-69dcb7821f9a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:48.746047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27788f-ee9e-45ec-9325-69dcb7821f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:48.746328 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27788f-ee9e-45ec-9325-69dcb7821f9a None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:48.747180 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27788f-ee9e-45ec-9325-69dcb7821f9a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:48.747496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27788f-ee9e-45ec-9325-69dcb7821f9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:48.748976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27788f-ee9e-45ec-9325-69dcb7821f9a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:48.768142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 458/904] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jun 17 07:07:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:48.773795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27788f-ee9e-45ec-9325-69dcb7821f9a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:48.793186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e94c7-5556-4dd9-a786-69e9c96c1de0 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:48.839850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240f0f9-ff77-4c1b-8350-8503096bbeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:48.841346 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240f0f9-ff77-4c1b-8350-8503096bbeee None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:48.841839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240f0f9-ff77-4c1b-8350-8503096bbeee None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:48.842306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240f0f9-ff77-4c1b-8350-8503096bbeee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:48.843388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240f0f9-ff77-4c1b-8350-8503096bbeee None admin] RBAC: Adding query filter params (interface=public, service_id=4acc363f22254f468259a9b4f1af5625, region_id=RegionOne) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:48.843861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240f0f9-ff77-4c1b-8350-8503096bbeee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:48.846427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240f0f9-ff77-4c1b-8350-8503096bbeee None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:48.854858 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 447/905] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jun 17 07:07:48 2022] GET /identity/v3/endpoints?service_id=4acc363f22254f468259a9b4f1af5625&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:48.859918 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240f0f9-ff77-4c1b-8350-8503096bbeee None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:48.908870 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27788f-ee9e-45ec-9325-69dcb7821f9a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:48.928471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:48.946226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7567797-1960-4e44-9a9c-23d625362e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:48.947399 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7567797-1960-4e44-9a9c-23d625362e75 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:48.947696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7567797-1960-4e44-9a9c-23d625362e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:48.947971 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7567797-1960-4e44-9a9c-23d625362e75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:48.978927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240f0f9-ff77-4c1b-8350-8503096bbeee None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:48.986855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:49.014617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98083b6a-db38-4b50-810b-669584779fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:49.015210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98083b6a-db38-4b50-810b-669584779fcd None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:49.015496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98083b6a-db38-4b50-810b-669584779fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:49.015774 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98083b6a-db38-4b50-810b-669584779fcd None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:49.016477 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98083b6a-db38-4b50-810b-669584779fcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:49.018070 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98083b6a-db38-4b50-810b-669584779fcd None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:49.024660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7567797-1960-4e44-9a9c-23d625362e75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:49.025325 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 448/906] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jun 17 07:07:48 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:07:49.027047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7567797-1960-4e44-9a9c-23d625362e75 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:49.142535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 459/907] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:48 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 07:07:49.148058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98083b6a-db38-4b50-810b-669584779fcd None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:49.249471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:49.259631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98083b6a-db38-4b50-810b-669584779fcd None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:49.295206 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e4068-0f50-498e-a7f1-858ef4582233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:49.295933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e4068-0f50-498e-a7f1-858ef4582233 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:49.296247 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e4068-0f50-498e-a7f1-858ef4582233 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:49.296527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e4068-0f50-498e-a7f1-858ef4582233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:49.308817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e4068-0f50-498e-a7f1-858ef4582233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:49.310275 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e4068-0f50-498e-a7f1-858ef4582233 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-cb6e4068-0f50-498e-a7f1-858ef4582233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:49.327882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:49.328861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:49.329728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:49.329728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:49.329728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:49.329728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:49.334067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 449/908] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:49.335430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7567797-1960-4e44-9a9c-23d625362e75 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:49.412355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:49.448833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7567797-1960-4e44-9a9c-23d625362e75 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:49.485918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcead9ae-68a9-4d05-a79e-196dcd4cdf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:49.486994 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcead9ae-68a9-4d05-a79e-196dcd4cdf5b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:49.487496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcead9ae-68a9-4d05-a79e-196dcd4cdf5b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:49.488019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcead9ae-68a9-4d05-a79e-196dcd4cdf5b None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:49.488994 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcead9ae-68a9-4d05-a79e-196dcd4cdf5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:49.489520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcead9ae-68a9-4d05-a79e-196dcd4cdf5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:49.491078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcead9ae-68a9-4d05-a79e-196dcd4cdf5b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:49.507938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 460/909] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:49.509723 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e4068-0f50-498e-a7f1-858ef4582233 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:49.614242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e4068-0f50-498e-a7f1-858ef4582233 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:49.651186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedfa49-5dc3-423e-a538-45ca38306678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:49.652376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedfa49-5dc3-423e-a538-45ca38306678 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:49.652896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedfa49-5dc3-423e-a538-45ca38306678 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:49.653388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedfa49-5dc3-423e-a538-45ca38306678 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:49.660442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baedfa49-5dc3-423e-a538-45ca38306678 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:49.662167 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baedfa49-5dc3-423e-a538-45ca38306678 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-baedfa49-5dc3-423e-a538-45ca38306678 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:49.677396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:49.678362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:49.679321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:49.679321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:49.679321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:49.679321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:49.679321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:49.679321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:49.679321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:49.679321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:49.686657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 450/910] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:49.688291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcead9ae-68a9-4d05-a79e-196dcd4cdf5b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:49.798441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcead9ae-68a9-4d05-a79e-196dcd4cdf5b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:49.830298 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:49.834753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6402c6-fb83-45fc-966c-4c2c681dbbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:49.835422 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6402c6-fb83-45fc-966c-4c2c681dbbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:49.835706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6402c6-fb83-45fc-966c-4c2c681dbbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:49.835984 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6402c6-fb83-45fc-966c-4c2c681dbbb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:49.836825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6402c6-fb83-45fc-966c-4c2c681dbbb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:49.837193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6402c6-fb83-45fc-966c-4c2c681dbbb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:49.839130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6402c6-fb83-45fc-966c-4c2c681dbbb6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:49.852214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 461/911] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:49.858252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedfa49-5dc3-423e-a538-45ca38306678 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:49.864925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:49.958635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200522c-8c1e-4a9e-9c5b-b6d49ab620dc None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:49.958869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200522c-8c1e-4a9e-9c5b-b6d49ab620dc None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:49.958960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200522c-8c1e-4a9e-9c5b-b6d49ab620dc None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:49.960761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 462/912] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:07:49.966359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a2d06-7e40-4923-a13e-cee827ad72ba None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:49.966514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a2d06-7e40-4923-a13e-cee827ad72ba None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:49.966794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a2d06-7e40-4923-a13e-cee827ad72ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:49.978369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedfa49-5dc3-423e-a538-45ca38306678 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:49.984876 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-422a2d06-7e40-4923-a13e-cee827ad72ba None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:49.986066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-422a2d06-7e40-4923-a13e-cee827ad72ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:50.023633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a387e-60c0-4f83-a6af-f06b5205d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:50.025250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a387e-60c0-4f83-a6af-f06b5205d23e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:50.025741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a387e-60c0-4f83-a6af-f06b5205d23e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:50.026242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a387e-60c0-4f83-a6af-f06b5205d23e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:50.027273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a387e-60c0-4f83-a6af-f06b5205d23e None admin] RBAC: Adding query filter params (role.id=b711a007d49d405fa3b1bf14eaa94b81, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=dc98526f14e84279b002c43ef44cabed) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:50.027823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a387e-60c0-4f83-a6af-f06b5205d23e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:50.029972 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a387e-60c0-4f83-a6af-f06b5205d23e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:50.038383 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 451/913] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:49 2022] GET /identity/v3/role_assignments?user.id=dc98526f14e84279b002c43ef44cabed&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=b711a007d49d405fa3b1bf14eaa94b81 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:50.059807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 463/914] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:50.083536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a387e-60c0-4f83-a6af-f06b5205d23e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:50.160696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:50.201859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a387e-60c0-4f83-a6af-f06b5205d23e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:50.238709 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb33ca4-9ae9-47dc-bbd1-e308ad575151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:50.239521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb33ca4-9ae9-47dc-bbd1-e308ad575151 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:50.239809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb33ca4-9ae9-47dc-bbd1-e308ad575151 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:50.240107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb33ca4-9ae9-47dc-bbd1-e308ad575151 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:50.240763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb33ca4-9ae9-47dc-bbd1-e308ad575151 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:50.242130 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb33ca4-9ae9-47dc-bbd1-e308ad575151 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-aeb33ca4-9ae9-47dc-bbd1-e308ad575151 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:50.255100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:50.256252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:50.257444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 452/915] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jun 17 07:07:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:50.265304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a2d06-7e40-4923-a13e-cee827ad72ba None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:50.370696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a2d06-7e40-4923-a13e-cee827ad72ba None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:50.392086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:50.408427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a8555-1685-4ff0-88b3-91016a281ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:50.409102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a8555-1685-4ff0-88b3-91016a281ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:50.409446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a8555-1685-4ff0-88b3-91016a281ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:50.409730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a8555-1685-4ff0-88b3-91016a281ed6 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:50.410487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a8555-1685-4ff0-88b3-91016a281ed6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:50.410856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a8555-1685-4ff0-88b3-91016a281ed6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:50.412200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a8555-1685-4ff0-88b3-91016a281ed6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:50.418073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 464/916] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jun 17 07:07:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:50.425180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb33ca4-9ae9-47dc-bbd1-e308ad575151 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:50.533807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb33ca4-9ae9-47dc-bbd1-e308ad575151 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:50.568649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fc619-2f9a-4ff0-9ab4-cd3c48f00c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:50.569736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fc619-2f9a-4ff0-9ab4-cd3c48f00c98 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:50.570235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:50.570781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fc619-2f9a-4ff0-9ab4-cd3c48f00c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:50.571208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fc619-2f9a-4ff0-9ab4-cd3c48f00c98 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:50.572805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fc619-2f9a-4ff0-9ab4-cd3c48f00c98 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:50.573268 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fc619-2f9a-4ff0-9ab4-cd3c48f00c98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:50.574848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fc619-2f9a-4ff0-9ab4-cd3c48f00c98 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:50.584417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 453/917] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jun 17 07:07:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:50.586595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a8555-1685-4ff0-88b3-91016a281ed6 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:50.695755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a8555-1685-4ff0-88b3-91016a281ed6 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:50.734397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159a0df-a55d-41cc-b8ff-0b833c032a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:50.735088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159a0df-a55d-41cc-b8ff-0b833c032a43 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:50.735383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159a0df-a55d-41cc-b8ff-0b833c032a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:50.735667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159a0df-a55d-41cc-b8ff-0b833c032a43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:50.736536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159a0df-a55d-41cc-b8ff-0b833c032a43 None admin] RBAC: Adding query filter params (interface=public, service_id=9c9677fb860a420498fb11b1b4d94509, region_id=RegionOne) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:50.736953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159a0df-a55d-41cc-b8ff-0b833c032a43 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:50.738384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159a0df-a55d-41cc-b8ff-0b833c032a43 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:50.745528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 465/918] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jun 17 07:07:50 2022] GET /identity/v3/endpoints?service_id=9c9677fb860a420498fb11b1b4d94509&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:50.751572 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fc619-2f9a-4ff0-9ab4-cd3c48f00c98 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:50.791589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603b6e2-e14b-449d-b70c-9274658fe384 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:50.791902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603b6e2-e14b-449d-b70c-9274658fe384 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:50.792192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603b6e2-e14b-449d-b70c-9274658fe384 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:50.793667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 466/919] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:07:50.799558 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc3ccf-86c5-4b96-9dad-c176bdd72822 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:50.799847 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc3ccf-86c5-4b96-9dad-c176bdd72822 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:50.800155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc3ccf-86c5-4b96-9dad-c176bdd72822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:50.823456 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-28cc3ccf-86c5-4b96-9dad-c176bdd72822 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:50.825068 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-28cc3ccf-86c5-4b96-9dad-c176bdd72822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:50.867411 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fc619-2f9a-4ff0-9ab4-cd3c48f00c98 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:50.874513 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:50.911603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffdc0f-a0c5-4e72-80aa-bc26598377c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:50.911603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffdc0f-a0c5-4e72-80aa-bc26598377c5 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:50.911603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffdc0f-a0c5-4e72-80aa-bc26598377c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:50.911603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffdc0f-a0c5-4e72-80aa-bc26598377c5 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:50.912003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffdc0f-a0c5-4e72-80aa-bc26598377c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:50.912003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffdc0f-a0c5-4e72-80aa-bc26598377c5 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:50.912500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 467/920] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:50.924195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 454/921] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jun 17 07:07:50 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:07:50.930178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc3ccf-86c5-4b96-9dad-c176bdd72822 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:51.041610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc3ccf-86c5-4b96-9dad-c176bdd72822 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:51.087462 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5d1aa-305c-4ae4-82af-5d88443dff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:51.088298 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5d1aa-305c-4ae4-82af-5d88443dff6b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:51.088584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5d1aa-305c-4ae4-82af-5d88443dff6b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:51.088863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5d1aa-305c-4ae4-82af-5d88443dff6b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:51.089518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5d1aa-305c-4ae4-82af-5d88443dff6b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:51.090882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5d1aa-305c-4ae4-82af-5d88443dff6b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-07f5d1aa-305c-4ae4-82af-5d88443dff6b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:51.102986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:51.104010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:51.104888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 468/922] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:07:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:51.112022 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffdc0f-a0c5-4e72-80aa-bc26598377c5 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:51.175938 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:51.217208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffdc0f-a0c5-4e72-80aa-bc26598377c5 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:51.255910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748f4ad-61e0-48ea-ad24-08b348bb28e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:51.256698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748f4ad-61e0-48ea-ad24-08b348bb28e7 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:51.256981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748f4ad-61e0-48ea-ad24-08b348bb28e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:51.257309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748f4ad-61e0-48ea-ad24-08b348bb28e7 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:51.258095 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748f4ad-61e0-48ea-ad24-08b348bb28e7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:51.258415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748f4ad-61e0-48ea-ad24-08b348bb28e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:51.260376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748f4ad-61e0-48ea-ad24-08b348bb28e7 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:51.266075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 455/923] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:07:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:51.271701 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5d1aa-305c-4ae4-82af-5d88443dff6b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:51.303293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79c1cb-7fd1-4c82-ba37-41b312b61889 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:51.303574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79c1cb-7fd1-4c82-ba37-41b312b61889 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:51.303781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79c1cb-7fd1-4c82-ba37-41b312b61889 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:51.305175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 456/924] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:51.310399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d421d-15b3-42b4-89bf-4455179b12c3 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:51.310737 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d421d-15b3-42b4-89bf-4455179b12c3 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:51.311020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d421d-15b3-42b4-89bf-4455179b12c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:51.316400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:51.331308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-154d421d-15b3-42b4-89bf-4455179b12c3 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:51.332781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-154d421d-15b3-42b4-89bf-4455179b12c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:51.402841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5d1aa-305c-4ae4-82af-5d88443dff6b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:51.408242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 457/925] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:51.439251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d421d-15b3-42b4-89bf-4455179b12c3 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:51.443136 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970c406-7f52-4baf-9305-ea148da75414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:51.444118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970c406-7f52-4baf-9305-ea148da75414 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:51.444608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970c406-7f52-4baf-9305-ea148da75414 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:51.445101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970c406-7f52-4baf-9305-ea148da75414 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:51.446051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970c406-7f52-4baf-9305-ea148da75414 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:51.446591 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970c406-7f52-4baf-9305-ea148da75414 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:51.448207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970c406-7f52-4baf-9305-ea148da75414 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:51.454069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 469/926] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jun 17 07:07:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:51.459771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970c406-7f52-4baf-9305-ea148da75414 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:51.499672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:51.548148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d421d-15b3-42b4-89bf-4455179b12c3 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:51.570142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970c406-7f52-4baf-9305-ea148da75414 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:51.573327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:51.590989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475609ea-f26d-40c4-99c4-4c92f9e12be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:51.592148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475609ea-f26d-40c4-99c4-4c92f9e12be6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:51.593123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475609ea-f26d-40c4-99c4-4c92f9e12be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:51.593763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475609ea-f26d-40c4-99c4-4c92f9e12be6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:51.594709 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475609ea-f26d-40c4-99c4-4c92f9e12be6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:51.597223 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475609ea-f26d-40c4-99c4-4c92f9e12be6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:51.607160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f21bb-b334-4dae-b11d-33dac1547cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:51.607900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f21bb-b334-4dae-b11d-33dac1547cd2 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:51.608280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f21bb-b334-4dae-b11d-33dac1547cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:51.608629 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f21bb-b334-4dae-b11d-33dac1547cd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:51.609440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f21bb-b334-4dae-b11d-33dac1547cd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:51.610864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f21bb-b334-4dae-b11d-33dac1547cd2 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-475609ea-f26d-40c4-99c4-4c92f9e12be6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:51.613899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:51.614819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 07:07:51.615705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:51.617582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 458/927] 10.222.0.33 () {56 vars in 1132 bytes} [Fri Jun 17 07:07:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:51.623951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475609ea-f26d-40c4-99c4-4c92f9e12be6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:51.639793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 470/928] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jun 17 07:07:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:07:51.756376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475609ea-f26d-40c4-99c4-4c92f9e12be6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:51.776897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:51.803581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fad8c-4fed-4171-9a4d-2869f9b0d50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:51.804668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fad8c-4fed-4171-9a4d-2869f9b0d50b None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:51.805199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fad8c-4fed-4171-9a4d-2869f9b0d50b None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:51.805876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fad8c-4fed-4171-9a4d-2869f9b0d50b None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:51.806755 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fad8c-4fed-4171-9a4d-2869f9b0d50b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:51.807216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fad8c-4fed-4171-9a4d-2869f9b0d50b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:51.808855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fad8c-4fed-4171-9a4d-2869f9b0d50b None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:51.814853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 459/929] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jun 17 07:07:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:51.820836 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f21bb-b334-4dae-b11d-33dac1547cd2 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:51.934616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f21bb-b334-4dae-b11d-33dac1547cd2 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:51.970079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021eb0b0-2ebd-4a92-8b70-58f845e3a488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:51.970794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021eb0b0-2ebd-4a92-8b70-58f845e3a488 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:51.971088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021eb0b0-2ebd-4a92-8b70-58f845e3a488 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:51.971367 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021eb0b0-2ebd-4a92-8b70-58f845e3a488 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:51.972090 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021eb0b0-2ebd-4a92-8b70-58f845e3a488 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:51.972420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021eb0b0-2ebd-4a92-8b70-58f845e3a488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:51.973779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021eb0b0-2ebd-4a92-8b70-58f845e3a488 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:51.979084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 471/930] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jun 17 07:07:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:51.984661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fad8c-4fed-4171-9a4d-2869f9b0d50b None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:52.086474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fad8c-4fed-4171-9a4d-2869f9b0d50b None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:52.124112 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c633b4-e03e-4b4b-8749-5c4d4efc99e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:52.124804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c633b4-e03e-4b4b-8749-5c4d4efc99e4 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:52.125092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c633b4-e03e-4b4b-8749-5c4d4efc99e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:52.125373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c633b4-e03e-4b4b-8749-5c4d4efc99e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:52.126125 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c633b4-e03e-4b4b-8749-5c4d4efc99e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:52.127521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c633b4-e03e-4b4b-8749-5c4d4efc99e4 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:52.152239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 460/931] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jun 17 07:07:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:52.295988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9e606-30a9-4668-837b-57b418a1f755 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:52.296363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9e606-30a9-4668-837b-57b418a1f755 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:52.296584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9e606-30a9-4668-837b-57b418a1f755 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:52.298204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 472/932] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:52.303958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403366ce-8fdc-4beb-90e5-82309864c989 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:52.304501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403366ce-8fdc-4beb-90e5-82309864c989 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:52.304598 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403366ce-8fdc-4beb-90e5-82309864c989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:52.323800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-403366ce-8fdc-4beb-90e5-82309864c989 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:52.325308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-403366ce-8fdc-4beb-90e5-82309864c989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:52.354436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:52.410646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 461/933] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:52.432966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9e606-30a9-4668-837b-57b418a1f755 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:52.459816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:52.538010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9e606-30a9-4668-837b-57b418a1f755 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:52.572795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade0afc-3e9c-46c5-84e1-142bfa0a002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:52.573613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade0afc-3e9c-46c5-84e1-142bfa0a002c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:52.573894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade0afc-3e9c-46c5-84e1-142bfa0a002c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:52.574170 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade0afc-3e9c-46c5-84e1-142bfa0a002c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:52.584501 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade0afc-3e9c-46c5-84e1-142bfa0a002c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-0ade0afc-3e9c-46c5-84e1-142bfa0a002c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:52.593271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:52.594206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:52.595047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 473/934] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jun 17 07:07:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:52.600886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403366ce-8fdc-4beb-90e5-82309864c989 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:52.706454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403366ce-8fdc-4beb-90e5-82309864c989 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:52.742863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953b1b3-3f11-4eda-9e69-c13d1d4f0cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:52.743569 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953b1b3-3f11-4eda-9e69-c13d1d4f0cda None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:52.743857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953b1b3-3f11-4eda-9e69-c13d1d4f0cda None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:52.744165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953b1b3-3f11-4eda-9e69-c13d1d4f0cda None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:52.744914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953b1b3-3f11-4eda-9e69-c13d1d4f0cda None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:52.745226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953b1b3-3f11-4eda-9e69-c13d1d4f0cda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:52.746602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953b1b3-3f11-4eda-9e69-c13d1d4f0cda None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:52.755317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 462/935] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jun 17 07:07:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:52.761066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade0afc-3e9c-46c5-84e1-142bfa0a002c None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:52.850242 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:52.868248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade0afc-3e9c-46c5-84e1-142bfa0a002c None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:52.905161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce0958-2811-45f3-b4a4-91f75f763e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:52.905965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce0958-2811-45f3-b4a4-91f75f763e49 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:52.906251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce0958-2811-45f3-b4a4-91f75f763e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:52.906534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce0958-2811-45f3-b4a4-91f75f763e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:52.918598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce0958-2811-45f3-b4a4-91f75f763e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:52.920054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce0958-2811-45f3-b4a4-91f75f763e49 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-78ce0958-2811-45f3-b4a4-91f75f763e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:52.938158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:52.939166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:52.940047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:52.940047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:52.940047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:52.940047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:52.940047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 474/936] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:52.944987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953b1b3-3f11-4eda-9e69-c13d1d4f0cda None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:52.971584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:53.057841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953b1b3-3f11-4eda-9e69-c13d1d4f0cda None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:53.093804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb18c12-0577-437e-9d66-cbdd476256b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:53.094479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb18c12-0577-437e-9d66-cbdd476256b4 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:53.094813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb18c12-0577-437e-9d66-cbdd476256b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:53.095099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb18c12-0577-437e-9d66-cbdd476256b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:53.095825 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb18c12-0577-437e-9d66-cbdd476256b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:53.096161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb18c12-0577-437e-9d66-cbdd476256b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:53.097545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb18c12-0577-437e-9d66-cbdd476256b4 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:53.110817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 463/937] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:53.116202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce0958-2811-45f3-b4a4-91f75f763e49 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:53.181860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416df42-d8a2-4236-b826-25600f48186b None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:53.182189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416df42-d8a2-4236-b826-25600f48186b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:53.182431 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416df42-d8a2-4236-b826-25600f48186b None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:53.183921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 464/938] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:53.190695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae5f40-e273-40de-b86b-5b3b7387976c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:53.190695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae5f40-e273-40de-b86b-5b3b7387976c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:53.190695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae5f40-e273-40de-b86b-5b3b7387976c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:53.206060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-e9ae5f40-e273-40de-b86b-5b3b7387976c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:53.207508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e9ae5f40-e273-40de-b86b-5b3b7387976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:53.243953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce0958-2811-45f3-b4a4-91f75f763e49 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:53.284030 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 465/939] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:53.287363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835c547-7b6e-4a18-81f4-ae1646a11f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:53.288222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835c547-7b6e-4a18-81f4-ae1646a11f1c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:53.288511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835c547-7b6e-4a18-81f4-ae1646a11f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:53.288794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835c547-7b6e-4a18-81f4-ae1646a11f1c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:53.291605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:53.299114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835c547-7b6e-4a18-81f4-ae1646a11f1c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:53.300747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835c547-7b6e-4a18-81f4-ae1646a11f1c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:53.304176 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae5f40-e273-40de-b86b-5b3b7387976c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-e835c547-7b6e-4a18-81f4-ae1646a11f1c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:53.320677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:53.321760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:53.322703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:53.322703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:53.322703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:53.322703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:53.322703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:53.322703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:53.322703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:53.322703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:53.322703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 07:07:53.322703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:53.322703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 475/940] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jun 17 07:07:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:53.323321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:53.327790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835c547-7b6e-4a18-81f4-ae1646a11f1c None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:53.429872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae5f40-e273-40de-b86b-5b3b7387976c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:53.438564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835c547-7b6e-4a18-81f4-ae1646a11f1c None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:53.467497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e931f-23cd-401c-8a70-929045f42140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:53.468706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e931f-23cd-401c-8a70-929045f42140 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:53.469166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e931f-23cd-401c-8a70-929045f42140 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:53.469609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e931f-23cd-401c-8a70-929045f42140 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:53.470434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e931f-23cd-401c-8a70-929045f42140 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:53.471991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e931f-23cd-401c-8a70-929045f42140 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:53.476450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1560c6-6df0-44ab-b2e0-16c1b0a3c46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:53.477202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1560c6-6df0-44ab-b2e0-16c1b0a3c46b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:53.477594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1560c6-6df0-44ab-b2e0-16c1b0a3c46b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:53.477940 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1560c6-6df0-44ab-b2e0-16c1b0a3c46b None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:53.478847 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1560c6-6df0-44ab-b2e0-16c1b0a3c46b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:53.479243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1560c6-6df0-44ab-b2e0-16c1b0a3c46b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:53.480759 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1560c6-6df0-44ab-b2e0-16c1b0a3c46b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-417e931f-23cd-401c-8a70-929045f42140 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:53.485961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:53.486961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:53.487796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:53.490704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 466/941] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jun 17 07:07:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:53.499114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e931f-23cd-401c-8a70-929045f42140 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:53.500362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 476/942] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jun 17 07:07:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:53.508179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1560c6-6df0-44ab-b2e0-16c1b0a3c46b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:53.598486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:53.624504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:53.631991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e931f-23cd-401c-8a70-929045f42140 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:53.636423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1560c6-6df0-44ab-b2e0-16c1b0a3c46b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:53.676848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2c01a-f44d-4ea2-bdfc-d689dc778228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:53.677720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2c01a-f44d-4ea2-bdfc-d689dc778228 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:53.678211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2c01a-f44d-4ea2-bdfc-d689dc778228 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:53.678649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2c01a-f44d-4ea2-bdfc-d689dc778228 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:53.679522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2c01a-f44d-4ea2-bdfc-d689dc778228 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:53.679901 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2c01a-f44d-4ea2-bdfc-d689dc778228 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:53.680423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431868bd-170b-4722-9993-3a87db090f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:53.681471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431868bd-170b-4722-9993-3a87db090f39 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:53.681578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2c01a-f44d-4ea2-bdfc-d689dc778228 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:53.682030 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431868bd-170b-4722-9993-3a87db090f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:53.682473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431868bd-170b-4722-9993-3a87db090f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:53.687902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 467/943] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jun 17 07:07:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:53.693509 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2c01a-f44d-4ea2-bdfc-d689dc778228 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:53.695984 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431868bd-170b-4722-9993-3a87db090f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:53.699475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431868bd-170b-4722-9993-3a87db090f39 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-431868bd-170b-4722-9993-3a87db090f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:53.722623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:53.723739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:53.724851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:53.724851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:53.724851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:53.724851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:53.732045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 477/944] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:53.732937 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431868bd-170b-4722-9993-3a87db090f39 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:53.825272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2c01a-f44d-4ea2-bdfc-d689dc778228 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:53.853973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431868bd-170b-4722-9993-3a87db090f39 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:53.862009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:53.863610 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857849b-7c57-4965-a6b8-6b97c11a6db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:53.864834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857849b-7c57-4965-a6b8-6b97c11a6db6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:53.865361 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857849b-7c57-4965-a6b8-6b97c11a6db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:53.865838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857849b-7c57-4965-a6b8-6b97c11a6db6 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:53.866807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857849b-7c57-4965-a6b8-6b97c11a6db6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:53.867496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857849b-7c57-4965-a6b8-6b97c11a6db6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:53.870537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857849b-7c57-4965-a6b8-6b97c11a6db6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:53.877197 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 468/945] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jun 17 07:07:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:53.882648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857849b-7c57-4965-a6b8-6b97c11a6db6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:53.889688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:53.904832 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aac727-f06a-4fb5-8179-e9f793063085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:53.905521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aac727-f06a-4fb5-8179-e9f793063085 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:53.905854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aac727-f06a-4fb5-8179-e9f793063085 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:53.906136 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aac727-f06a-4fb5-8179-e9f793063085 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:53.907196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aac727-f06a-4fb5-8179-e9f793063085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:53.907512 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aac727-f06a-4fb5-8179-e9f793063085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:53.909535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aac727-f06a-4fb5-8179-e9f793063085 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:53.928212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 478/946] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:53.931772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aac727-f06a-4fb5-8179-e9f793063085 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:54.006956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857849b-7c57-4965-a6b8-6b97c11a6db6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:54.038037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aac727-f06a-4fb5-8179-e9f793063085 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:54.041917 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead100f-16e2-4540-8945-de8272c5b658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:54.042582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead100f-16e2-4540-8945-de8272c5b658 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:54.042870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead100f-16e2-4540-8945-de8272c5b658 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:54.043151 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead100f-16e2-4540-8945-de8272c5b658 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:54.043892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead100f-16e2-4540-8945-de8272c5b658 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:54.045303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead100f-16e2-4540-8945-de8272c5b658 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:54.066680 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 469/947] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jun 17 07:07:53 2022] POST /identity/v3/endpoints => generated 349 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:07:54.068495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df91cf-bf85-4097-b7d8-a1ce51e0ef59 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:54.069142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df91cf-bf85-4097-b7d8-a1ce51e0ef59 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:54.070035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df91cf-bf85-4097-b7d8-a1ce51e0ef59 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:54.072502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 470/948] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:54.078639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275872d-8532-4059-aecd-d9a2a2979598 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:54.080244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275872d-8532-4059-aecd-d9a2a2979598 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:54.081543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275872d-8532-4059-aecd-d9a2a2979598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:54.089418 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee50488-fa65-4240-ba80-969682d1b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:54.090662 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee50488-fa65-4240-ba80-969682d1b911 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:54.091158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee50488-fa65-4240-ba80-969682d1b911 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:54.091635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee50488-fa65-4240-ba80-969682d1b911 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:54.100268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee50488-fa65-4240-ba80-969682d1b911 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:54.101669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-a275872d-8532-4059-aecd-d9a2a2979598 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:54.102373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee50488-fa65-4240-ba80-969682d1b911 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:54.103908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-a275872d-8532-4059-aecd-d9a2a2979598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-aee50488-fa65-4240-ba80-969682d1b911 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:54.122844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:54.123859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:54.124782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:54.124782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:54.124782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:54.124782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:54.124782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:54.124782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:54.124782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:54.124782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:54.132313 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 479/949] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:54.133596 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee50488-fa65-4240-ba80-969682d1b911 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:54.161701 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:54.172214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:54.192581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 471/950] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:54.211175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275872d-8532-4059-aecd-d9a2a2979598 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:54.251381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee50488-fa65-4240-ba80-969682d1b911 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:54.289153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d45e07-ca14-4c8c-827b-a6ad332b10ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:54.290475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d45e07-ca14-4c8c-827b-a6ad332b10ee None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:54.290872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d45e07-ca14-4c8c-827b-a6ad332b10ee None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:54.291254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d45e07-ca14-4c8c-827b-a6ad332b10ee None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:54.292164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d45e07-ca14-4c8c-827b-a6ad332b10ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:54.292573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d45e07-ca14-4c8c-827b-a6ad332b10ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:54.294165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d45e07-ca14-4c8c-827b-a6ad332b10ee None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:54.308850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 480/951] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:54.311671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d45e07-ca14-4c8c-827b-a6ad332b10ee None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:54.328838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275872d-8532-4059-aecd-d9a2a2979598 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:54.369805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba54c0-d6c1-4ec8-a4c9-ab1db5309dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:54.370841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba54c0-d6c1-4ec8-a4c9-ab1db5309dbb None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:54.371388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba54c0-d6c1-4ec8-a4c9-ab1db5309dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:54.371668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba54c0-d6c1-4ec8-a4c9-ab1db5309dbb None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:54.372433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba54c0-d6c1-4ec8-a4c9-ab1db5309dbb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:54.374350 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba54c0-d6c1-4ec8-a4c9-ab1db5309dbb None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:54.375727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: WARNING keystone.server.flask.common [None req-d8ba54c0-d6c1-4ec8-a4c9-ab1db5309dbb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 17 07:07:54.405209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.sql.core [None req-d8ba54c0-d6c1-4ec8-a4c9-ab1db5309dbb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 17 07:07:54.405209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 17 07:07:54.405209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [parameters: {'id': '4c845a842e48469baa88740e7e0990e5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 17 07:07:54.405209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80118) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-d8ba54c0-d6c1-4ec8-a4c9-ab1db5309dbb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4c845a842e48469baa88740e7e0990e5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4c845a842e48469baa88740e7e0990e5. Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application conn.query(q) Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 07:07:54.422495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application result.read() Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 17 07:07:54.423388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application next(self.gen) Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application yield resource Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application next(self.gen) Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self.session.rollback() Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self.force_reraise() Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise self.value Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application session.commit() Jun 17 07:07:54.424389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._prepare_impl() Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self.session.flush() Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._flush(objects) Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application compat.raise_( Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application flush_context.execute() Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 17 07:07:54.425299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rec.execute(self) Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 17 07:07:54.426217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application conn.query(q) Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application result.read() Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 07:07:54.427036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [parameters: {'id': '4c845a842e48469baa88740e7e0990e5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:54.427911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:54.430713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:54.432841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:54.432841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 17 07:07:54.432841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 17 07:07:54.432841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:54.432841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:54.432841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 17 07:07:54.432841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 17 07:07:54.432841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4c845a842e48469baa88740e7e0990e5. Jun 17 07:07:54.432841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:54.432841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 472/952] 10.222.0.33 () {60 vars in 1140 bytes} [Fri Jun 17 07:07:54 2022] POST /identity/v3/projects => generated 262 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:54.432841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba54c0-d6c1-4ec8-a4c9-ab1db5309dbb None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:54.439512 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:54.473844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d45e07-ca14-4c8c-827b-a6ad332b10ee None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:54.490445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:54.525932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf7418-aaa8-469c-b300-dbe49ffd123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:54.527182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf7418-aaa8-469c-b300-dbe49ffd123b None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:54.527508 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf7418-aaa8-469c-b300-dbe49ffd123b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:54.527837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf7418-aaa8-469c-b300-dbe49ffd123b None admin] PATH_INFO: `/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/dc98526f14e84279b002c43ef44cabed/roles/b711a007d49d405fa3b1bf14eaa94b81` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:54.530065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf7418-aaa8-469c-b300-dbe49ffd123b None admin] RBAC: Authorizing `identity:create_grant(project_id=a4ec9b09302d4c918ef638d794468ba3, user_id=dc98526f14e84279b002c43ef44cabed, role_id=b711a007d49d405fa3b1bf14eaa94b81)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:54.531486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf7418-aaa8-469c-b300-dbe49ffd123b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:54.549485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba54c0-d6c1-4ec8-a4c9-ab1db5309dbb None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:54.552313 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 481/953] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:07:54 2022] PUT /identity/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/dc98526f14e84279b002c43ef44cabed/roles/b711a007d49d405fa3b1bf14eaa94b81 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:07:54.553878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29176a4-425e-44b0-8d70-0a198929b84d None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:54.554165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29176a4-425e-44b0-8d70-0a198929b84d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:54.554439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29176a4-425e-44b0-8d70-0a198929b84d None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:54.560114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 482/954] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:54.562369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ff005-9d19-4520-81c4-81ca56ddf44c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:54.562660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ff005-9d19-4520-81c4-81ca56ddf44c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:54.562936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ff005-9d19-4520-81c4-81ca56ddf44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:54.592870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3b17b-a9d4-41f3-a38a-98524e9b28f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:54.593669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3b17b-a9d4-41f3-a38a-98524e9b28f7 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:54.593965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3b17b-a9d4-41f3-a38a-98524e9b28f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:54.594249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3b17b-a9d4-41f3-a38a-98524e9b28f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:54.596836 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-b90ff005-9d19-4520-81c4-81ca56ddf44c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:54.598884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b90ff005-9d19-4520-81c4-81ca56ddf44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:54.602357 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3b17b-a9d4-41f3-a38a-98524e9b28f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:54.604288 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3b17b-a9d4-41f3-a38a-98524e9b28f7 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-93a3b17b-a9d4-41f3-a38a-98524e9b28f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:54.621290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:54.622368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:54.623780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:54.623780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:54.623780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:54.623780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:54.623780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:54.623780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:54.623780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:54.623780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:54.623780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 473/955] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:54.628800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3b17b-a9d4-41f3-a38a-98524e9b28f7 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:54.700275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 483/956] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:54.722441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ff005-9d19-4520-81c4-81ca56ddf44c None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:54.760642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3b17b-a9d4-41f3-a38a-98524e9b28f7 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:54.768902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:54.803828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550ac56-b226-4c61-9620-5a75dd277b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:54.804506 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550ac56-b226-4c61-9620-5a75dd277b59 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:54.804789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550ac56-b226-4c61-9620-5a75dd277b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:54.805066 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550ac56-b226-4c61-9620-5a75dd277b59 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:54.806446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550ac56-b226-4c61-9620-5a75dd277b59 None admin] RBAC: Adding query filter params (name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:54.806756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550ac56-b226-4c61-9620-5a75dd277b59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:54.808269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550ac56-b226-4c61-9620-5a75dd277b59 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:54.808767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:54.836911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 474/957] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:07:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:54.848031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ff005-9d19-4520-81c4-81ca56ddf44c None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:54.887126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4414b-9059-4ae4-8cc7-208579a5bc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:54.887939 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4414b-9059-4ae4-8cc7-208579a5bc28 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:54.888250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4414b-9059-4ae4-8cc7-208579a5bc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:54.888529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4414b-9059-4ae4-8cc7-208579a5bc28 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:54.889190 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4414b-9059-4ae4-8cc7-208579a5bc28 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:54.890562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4414b-9059-4ae4-8cc7-208579a5bc28 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-ced4414b-9059-4ae4-8cc7-208579a5bc28 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:54.903580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:54.904791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 07:07:54.905669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:54.909957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 484/958] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:54.911437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550ac56-b226-4c61-9620-5a75dd277b59 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:55.034583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550ac56-b226-4c61-9620-5a75dd277b59 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:55.072300 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c0177-b4bc-4baa-b4ea-09e5a8757b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:55.072979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c0177-b4bc-4baa-b4ea-09e5a8757b9a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:55.073262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c0177-b4bc-4baa-b4ea-09e5a8757b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:55.073538 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c0177-b4bc-4baa-b4ea-09e5a8757b9a None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:55.074250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c0177-b4bc-4baa-b4ea-09e5a8757b9a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:55.074558 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c0177-b4bc-4baa-b4ea-09e5a8757b9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:55.075921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c0177-b4bc-4baa-b4ea-09e5a8757b9a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:55.081653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 475/959] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:07:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:55.087187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4414b-9059-4ae4-8cc7-208579a5bc28 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:55.196181 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4414b-9059-4ae4-8cc7-208579a5bc28 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:55.231314 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f462065-e442-402b-9a98-e586163e5255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:55.232394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f462065-e442-402b-9a98-e586163e5255 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:55.232884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f462065-e442-402b-9a98-e586163e5255 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:55.233385 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f462065-e442-402b-9a98-e586163e5255 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:55.234299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f462065-e442-402b-9a98-e586163e5255 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:55.234807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f462065-e442-402b-9a98-e586163e5255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:55.236402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f462065-e442-402b-9a98-e586163e5255 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:55.238877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:55.242700 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 485/960] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:07:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:56.330424 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83a5d9-8c90-4378-97be-1f879dd57270 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:56.330976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83a5d9-8c90-4378-97be-1f879dd57270 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:56.330976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83a5d9-8c90-4378-97be-1f879dd57270 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:56.332737 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 476/961] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:56.339387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3004dc-5265-4588-9c75-96d2f8ab0b47 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:56.339806 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3004dc-5265-4588-9c75-96d2f8ab0b47 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:56.339983 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3004dc-5265-4588-9c75-96d2f8ab0b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:56.357552 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-af3004dc-5265-4588-9c75-96d2f8ab0b47 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:56.358730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-af3004dc-5265-4588-9c75-96d2f8ab0b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:56.428040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 486/962] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:56.450802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83a5d9-8c90-4378-97be-1f879dd57270 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:56.482388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:56.555255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83a5d9-8c90-4378-97be-1f879dd57270 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:56.590263 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db32b2-9f61-4096-b8ce-ae20e7487e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:56.590985 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db32b2-9f61-4096-b8ce-ae20e7487e8e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:56.591274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db32b2-9f61-4096-b8ce-ae20e7487e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:56.591551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db32b2-9f61-4096-b8ce-ae20e7487e8e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:56.592226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db32b2-9f61-4096-b8ce-ae20e7487e8e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:56.593599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db32b2-9f61-4096-b8ce-ae20e7487e8e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-87db32b2-9f61-4096-b8ce-ae20e7487e8e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:56.606141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:56.607146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:56.608002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 477/963] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jun 17 07:07:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:56.613704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3004dc-5265-4588-9c75-96d2f8ab0b47 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:56.719658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3004dc-5265-4588-9c75-96d2f8ab0b47 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:56.752965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5654928-3785-4fee-8334-aa52cc16a3ca None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:56.753419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5654928-3785-4fee-8334-aa52cc16a3ca None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:56.753807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5654928-3785-4fee-8334-aa52cc16a3ca None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:56.754915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e24539-7a6a-4a9a-98d6-2128f5ab01a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:56.755584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e24539-7a6a-4a9a-98d6-2128f5ab01a1 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:56.755873 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e24539-7a6a-4a9a-98d6-2128f5ab01a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:56.756184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e24539-7a6a-4a9a-98d6-2128f5ab01a1 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:56.756633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 478/964] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:56.756899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e24539-7a6a-4a9a-98d6-2128f5ab01a1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:56.757209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e24539-7a6a-4a9a-98d6-2128f5ab01a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:56.759130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e24539-7a6a-4a9a-98d6-2128f5ab01a1 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:56.760277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:56.761767 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9404a3d-d41b-4f8d-a46d-56002fbc7332 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:56.762060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9404a3d-d41b-4f8d-a46d-56002fbc7332 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:56.762347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9404a3d-d41b-4f8d-a46d-56002fbc7332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:56.764597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 487/965] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jun 17 07:07:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:56.770194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e24539-7a6a-4a9a-98d6-2128f5ab01a1 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:56.782499 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-b9404a3d-d41b-4f8d-a46d-56002fbc7332 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:56.784281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b9404a3d-d41b-4f8d-a46d-56002fbc7332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:56.856250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 479/966] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:07:56.882877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9404a3d-d41b-4f8d-a46d-56002fbc7332 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:56.897752 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e24539-7a6a-4a9a-98d6-2128f5ab01a1 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:56.916411 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:56.944973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d12ad1-c350-4d4a-a808-b8a9272cd02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:56.945839 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d12ad1-c350-4d4a-a808-b8a9272cd02e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:56.946226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d12ad1-c350-4d4a-a808-b8a9272cd02e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:56.946625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d12ad1-c350-4d4a-a808-b8a9272cd02e None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:56.947918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d12ad1-c350-4d4a-a808-b8a9272cd02e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:56.948497 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d12ad1-c350-4d4a-a808-b8a9272cd02e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:56.950310 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d12ad1-c350-4d4a-a808-b8a9272cd02e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:56.958123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 488/967] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jun 17 07:07:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:56.964543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d12ad1-c350-4d4a-a808-b8a9272cd02e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:57.001530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9404a3d-d41b-4f8d-a46d-56002fbc7332 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:57.037819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bc432-6485-40bf-8b03-03e86e2c527c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:57.038838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bc432-6485-40bf-8b03-03e86e2c527c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.039131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bc432-6485-40bf-8b03-03e86e2c527c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.039411 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bc432-6485-40bf-8b03-03e86e2c527c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.041043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:57.052154 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bc432-6485-40bf-8b03-03e86e2c527c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-503bc432-6485-40bf-8b03-03e86e2c527c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:07:57.061571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:57.062722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:57.063749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:57.063749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:57.063749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:57.063749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:57.063749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:07:57.063749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:07:57.063749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:07:57.063749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:07:57.063749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:07:57.063749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:07:57.063749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 07:07:57.063749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:57.066239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 480/968] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jun 17 07:07:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:57.073154 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bc432-6485-40bf-8b03-03e86e2c527c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:57.100264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d12ad1-c350-4d4a-a808-b8a9272cd02e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:57.147788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d2451-dac3-4610-a523-75c49d34d08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:57.148427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2451-dac3-4610-a523-75c49d34d08d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.148722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2451-dac3-4610-a523-75c49d34d08d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.149153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2451-dac3-4610-a523-75c49d34d08d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.150743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d2451-dac3-4610-a523-75c49d34d08d None admin] RBAC: Adding query filter params (interface=admin, service_id=9c9677fb860a420498fb11b1b4d94509, region_id=RegionOne) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:57.151065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d2451-dac3-4610-a523-75c49d34d08d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:57.153260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d2451-dac3-4610-a523-75c49d34d08d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:57.166343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 489/969] 10.222.0.33 () {56 vars in 1249 bytes} [Fri Jun 17 07:07:56 2022] GET /identity/v3/endpoints?service_id=9c9677fb860a420498fb11b1b4d94509&interface=admin®ion_id=RegionOne => generated 193 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:57.172550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc040d-9b17-4482-9725-c8a2cc5ada93 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.173040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc040d-9b17-4482-9725-c8a2cc5ada93 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.173327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc040d-9b17-4482-9725-c8a2cc5ada93 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.174866 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 490/970] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:57.176571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc040d-9b17-4482-9725-c8a2cc5ada93 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:57.182127 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:57.203103 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bc432-6485-40bf-8b03-03e86e2c527c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:57.239033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f272f0-291a-4c72-9c8e-44b17cca56e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:57.239706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f272f0-291a-4c72-9c8e-44b17cca56e2 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.239993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f272f0-291a-4c72-9c8e-44b17cca56e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.240302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f272f0-291a-4c72-9c8e-44b17cca56e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.241029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f272f0-291a-4c72-9c8e-44b17cca56e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:57.241342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f272f0-291a-4c72-9c8e-44b17cca56e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:57.242714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f272f0-291a-4c72-9c8e-44b17cca56e2 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:57.251550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 481/971] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jun 17 07:07:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:57.252862 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6268777-9076-4943-83b6-24f7bb959066 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.253153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6268777-9076-4943-83b6-24f7bb959066 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.253432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6268777-9076-4943-83b6-24f7bb959066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.269671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c6268777-9076-4943-83b6-24f7bb959066 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:57.270843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c6268777-9076-4943-83b6-24f7bb959066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:57.291457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc040d-9b17-4482-9725-c8a2cc5ada93 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:57.324597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:57.339250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f438f07-7400-424f-b08d-912414f33181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:57.340176 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f438f07-7400-424f-b08d-912414f33181 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.340771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f438f07-7400-424f-b08d-912414f33181 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.340771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f438f07-7400-424f-b08d-912414f33181 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.341390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f438f07-7400-424f-b08d-912414f33181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:57.342749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f438f07-7400-424f-b08d-912414f33181 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:57.346691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 482/972] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:57.347460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6268777-9076-4943-83b6-24f7bb959066 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:57.351789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 491/973] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jun 17 07:07:57 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:07:57.359134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5d558-679a-4432-bb97-15fc616937f2 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.359425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5d558-679a-4432-bb97-15fc616937f2 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.359704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5d558-679a-4432-bb97-15fc616937f2 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.364143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 492/974] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:07:57.366597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79659ae9-257c-486d-ae6c-45e3302b14a1 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.366881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79659ae9-257c-486d-ae6c-45e3302b14a1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.367153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79659ae9-257c-486d-ae6c-45e3302b14a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.389599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-79659ae9-257c-486d-ae6c-45e3302b14a1 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:57.391198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-79659ae9-257c-486d-ae6c-45e3302b14a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:57.449071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:57.470244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 493/975] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:57.471096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79659ae9-257c-486d-ae6c-45e3302b14a1 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:57.483820 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6268777-9076-4943-83b6-24f7bb959066 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:57.521684 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0237fd1-251c-48dc-b692-11a7abb56f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:57.522498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0237fd1-251c-48dc-b692-11a7abb56f3e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.522788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0237fd1-251c-48dc-b692-11a7abb56f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.523069 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0237fd1-251c-48dc-b692-11a7abb56f3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.536049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0237fd1-251c-48dc-b692-11a7abb56f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:57.537617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0237fd1-251c-48dc-b692-11a7abb56f3e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-d0237fd1-251c-48dc-b692-11a7abb56f3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:57.555673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:57.556822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:57.557942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:57.557942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:57.557942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:57.557942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:57.563309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 483/976] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:57.563309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0237fd1-251c-48dc-b692-11a7abb56f3e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:57.598371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79659ae9-257c-486d-ae6c-45e3302b14a1 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:57.616343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:57.637049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356da13e-62ee-4a1c-bc6c-abf0eb07e022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:57.637823 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356da13e-62ee-4a1c-bc6c-abf0eb07e022 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.638211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356da13e-62ee-4a1c-bc6c-abf0eb07e022 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.638590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356da13e-62ee-4a1c-bc6c-abf0eb07e022 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.639385 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356da13e-62ee-4a1c-bc6c-abf0eb07e022 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:57.640961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356da13e-62ee-4a1c-bc6c-abf0eb07e022 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:57.642398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: WARNING keystone.server.flask.common [None req-356da13e-62ee-4a1c-bc6c-abf0eb07e022 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 17 07:07:57.655525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.sql.core [None req-356da13e-62ee-4a1c-bc6c-abf0eb07e022 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 17 07:07:57.655525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 17 07:07:57.655525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [parameters: {'user_id': '418f52dcdefb4d7fa202af0a9153f69f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 17 07:07:57.655525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80117) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-356da13e-62ee-4a1c-bc6c-abf0eb07e022 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application conn.query(q) Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application result.read() Jun 17 07:07:57.663755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 07:07:57.664907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application next(self.gen) Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application yield resource Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application next(self.gen) Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self.session.rollback() Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self.force_reraise() Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise self.value Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application session.commit() Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 17 07:07:57.671371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._prepare_impl() Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self.session.flush() Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._flush(objects) Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application compat.raise_( Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application flush_context.execute() Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rec.execute(self) Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 17 07:07:57.672504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 07:07:57.673528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application conn.query(q) Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application result.read() Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 07:07:57.674601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [parameters: {'user_id': '418f52dcdefb4d7fa202af0a9153f69f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:57.675770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 17 07:07:57.676822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 17 07:07:57.678179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 17 07:07:57.678179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 17 07:07:57.678179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 17 07:07:57.678179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 17 07:07:57.678179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 17 07:07:57.678179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:57.678179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 494/977] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jun 17 07:07:57 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:57.678179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356da13e-62ee-4a1c-bc6c-abf0eb07e022 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:57.684619 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0237fd1-251c-48dc-b692-11a7abb56f3e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:57.715640 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:57.724292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa95e0-4724-4b6e-adc4-84b04a2aeed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:57.724969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa95e0-4724-4b6e-adc4-84b04a2aeed2 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.725257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa95e0-4724-4b6e-adc4-84b04a2aeed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.725540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa95e0-4724-4b6e-adc4-84b04a2aeed2 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.726263 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa95e0-4724-4b6e-adc4-84b04a2aeed2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:57.728231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa95e0-4724-4b6e-adc4-84b04a2aeed2 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:57.742004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 484/978] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:07:57 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:07:57.744968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa95e0-4724-4b6e-adc4-84b04a2aeed2 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:57.798378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356da13e-62ee-4a1c-bc6c-abf0eb07e022 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:57.864530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96faf329-ad50-4ecc-b974-46edb8d4bec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:57.866180 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96faf329-ad50-4ecc-b974-46edb8d4bec2 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.866729 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96faf329-ad50-4ecc-b974-46edb8d4bec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.867160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96faf329-ad50-4ecc-b974-46edb8d4bec2 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.868383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96faf329-ad50-4ecc-b974-46edb8d4bec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:57.869342 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96faf329-ad50-4ecc-b974-46edb8d4bec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:57.871399 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96faf329-ad50-4ecc-b974-46edb8d4bec2 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:57.872808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa95e0-4724-4b6e-adc4-84b04a2aeed2 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:57.890037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 495/979] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:57.902543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96faf329-ad50-4ecc-b974-46edb8d4bec2 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:57.922516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594ab98-bb43-440e-8f0f-769df4122c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:57.923411 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594ab98-bb43-440e-8f0f-769df4122c10 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:57.923740 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594ab98-bb43-440e-8f0f-769df4122c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:57.927347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594ab98-bb43-440e-8f0f-769df4122c10 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:57.935258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594ab98-bb43-440e-8f0f-769df4122c10 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:57.936859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594ab98-bb43-440e-8f0f-769df4122c10 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:57.949966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-3594ab98-bb43-440e-8f0f-769df4122c10 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:57.958042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:57.959074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:57.960051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:57.960051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:57.960051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:57.960051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:57.960051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:57.960051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:57.960051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:57.960051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:57.960051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 07:07:57.960051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:57.960051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 485/980] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:07:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:57.965784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594ab98-bb43-440e-8f0f-769df4122c10 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:58.031123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96faf329-ad50-4ecc-b974-46edb8d4bec2 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:58.034530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:58.068485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b9078-c466-4a30-bcbd-425250e21b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:58.069338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b9078-c466-4a30-bcbd-425250e21b8b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:58.069621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b9078-c466-4a30-bcbd-425250e21b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:58.069904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b9078-c466-4a30-bcbd-425250e21b8b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:58.078242 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b9078-c466-4a30-bcbd-425250e21b8b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:58.079676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b9078-c466-4a30-bcbd-425250e21b8b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-f83b9078-c466-4a30-bcbd-425250e21b8b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:07:58.095773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:58.096807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:58.097790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:58.097790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:58.097790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:07:58.097790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:07:58.097790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:07:58.097790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:07:58.097790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:07:58.097790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:07:58.097790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 07:07:58.097790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:58.097790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 496/981] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jun 17 07:07:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:07:58.102729 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b9078-c466-4a30-bcbd-425250e21b8b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:58.115122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594ab98-bb43-440e-8f0f-769df4122c10 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:58.159810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27698bd4-b367-476e-841e-bfdb0e57fe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:58.160986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27698bd4-b367-476e-841e-bfdb0e57fe44 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:58.161384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27698bd4-b367-476e-841e-bfdb0e57fe44 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:58.161770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27698bd4-b367-476e-841e-bfdb0e57fe44 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:58.162704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27698bd4-b367-476e-841e-bfdb0e57fe44 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:58.163191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27698bd4-b367-476e-841e-bfdb0e57fe44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:58.165619 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27698bd4-b367-476e-841e-bfdb0e57fe44 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:58.210807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 486/982] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jun 17 07:07:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:58.212500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:58.224689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b9078-c466-4a30-bcbd-425250e21b8b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:58.270931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebeada-a8cf-40e3-a49b-fb7b6860e872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:58.271675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebeada-a8cf-40e3-a49b-fb7b6860e872 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:58.271970 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebeada-a8cf-40e3-a49b-fb7b6860e872 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:58.272327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebeada-a8cf-40e3-a49b-fb7b6860e872 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:58.273195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebeada-a8cf-40e3-a49b-fb7b6860e872 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:58.273496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebeada-a8cf-40e3-a49b-fb7b6860e872 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:58.275799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebeada-a8cf-40e3-a49b-fb7b6860e872 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:58.296550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 497/983] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jun 17 07:07:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:58.302456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27698bd4-b367-476e-841e-bfdb0e57fe44 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:58.413739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27698bd4-b367-476e-841e-bfdb0e57fe44 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:58.422520 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:58.452691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307b5f6-14df-490d-b10f-8513f3ae02e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:58.453398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307b5f6-14df-490d-b10f-8513f3ae02e5 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:58.453665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307b5f6-14df-490d-b10f-8513f3ae02e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:58.453949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307b5f6-14df-490d-b10f-8513f3ae02e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:58.532463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307b5f6-14df-490d-b10f-8513f3ae02e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:58.533847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307b5f6-14df-490d-b10f-8513f3ae02e5 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:58.633446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 487/984] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:07:58 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:07:58.639324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebeada-a8cf-40e3-a49b-fb7b6860e872 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:58.739189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebeada-a8cf-40e3-a49b-fb7b6860e872 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:58.772116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1ecbd-46ff-4e7c-82b0-194fab256702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:58.772899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1ecbd-46ff-4e7c-82b0-194fab256702 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:58.773185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1ecbd-46ff-4e7c-82b0-194fab256702 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:58.773461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1ecbd-46ff-4e7c-82b0-194fab256702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:58.783633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1ecbd-46ff-4e7c-82b0-194fab256702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:58.785039 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1ecbd-46ff-4e7c-82b0-194fab256702 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-c8b1ecbd-46ff-4e7c-82b0-194fab256702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:07:58.802585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:58.803458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:58.804387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:07:58.804387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:07:58.804387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:07:58.804387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:58.808130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 498/985] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:58.809843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307b5f6-14df-490d-b10f-8513f3ae02e5 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:58.818450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:58.911076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307b5f6-14df-490d-b10f-8513f3ae02e5 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:58.947654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d2dd1-7e14-4bc3-9669-1537aae05dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:58.948508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d2dd1-7e14-4bc3-9669-1537aae05dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:58.948891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d2dd1-7e14-4bc3-9669-1537aae05dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:58.949282 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d2dd1-7e14-4bc3-9669-1537aae05dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:58.950094 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d2dd1-7e14-4bc3-9669-1537aae05dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:58.950499 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d2dd1-7e14-4bc3-9669-1537aae05dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:58.951952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d2dd1-7e14-4bc3-9669-1537aae05dd7 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:58.968052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 488/986] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:07:58.969263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1ecbd-46ff-4e7c-82b0-194fab256702 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:58.973139 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:59.071617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1ecbd-46ff-4e7c-82b0-194fab256702 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:59.105018 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183cf6d-aaa1-43e9-b6d3-ec8114c78d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:59.105958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183cf6d-aaa1-43e9-b6d3-ec8114c78d68 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:59.106342 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183cf6d-aaa1-43e9-b6d3-ec8114c78d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:59.106713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183cf6d-aaa1-43e9-b6d3-ec8114c78d68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:59.113316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183cf6d-aaa1-43e9-b6d3-ec8114c78d68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:59.114839 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183cf6d-aaa1-43e9-b6d3-ec8114c78d68 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-c183cf6d-aaa1-43e9-b6d3-ec8114c78d68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:07:59.129335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:59.130500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:59.131606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:07:59.131606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:07:59.131606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:07:59.131606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:07:59.131606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:07:59.131606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:07:59.131606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:07:59.131606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:07:59.133127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 499/987] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:07:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:59.139097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d2dd1-7e14-4bc3-9669-1537aae05dd7 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:59.204582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:59.240141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d2dd1-7e14-4bc3-9669-1537aae05dd7 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:59.273721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb677d33-620b-46c3-9e65-ecf8324cc2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:59.275298 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb677d33-620b-46c3-9e65-ecf8324cc2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:59.275676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb677d33-620b-46c3-9e65-ecf8324cc2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:59.276050 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb677d33-620b-46c3-9e65-ecf8324cc2c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:59.276950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb677d33-620b-46c3-9e65-ecf8324cc2c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:59.277358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb677d33-620b-46c3-9e65-ecf8324cc2c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:59.278863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb677d33-620b-46c3-9e65-ecf8324cc2c8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:59.296209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 489/988] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:07:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:59.297276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183cf6d-aaa1-43e9-b6d3-ec8114c78d68 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:59.358615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:59.404687 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183cf6d-aaa1-43e9-b6d3-ec8114c78d68 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:59.440465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693d7fb-d653-4f4d-b4d1-6f9bff0c2228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:59.441151 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693d7fb-d653-4f4d-b4d1-6f9bff0c2228 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:59.441434 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693d7fb-d653-4f4d-b4d1-6f9bff0c2228 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:59.441708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693d7fb-d653-4f4d-b4d1-6f9bff0c2228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:59.442567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693d7fb-d653-4f4d-b4d1-6f9bff0c2228 None admin] RBAC: Adding query filter params (role.id=b711a007d49d405fa3b1bf14eaa94b81, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=dc98526f14e84279b002c43ef44cabed) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:59.442889 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693d7fb-d653-4f4d-b4d1-6f9bff0c2228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:59.444404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693d7fb-d653-4f4d-b4d1-6f9bff0c2228 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:59.456914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 500/989] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:07:59 2022] GET /identity/v3/role_assignments?user.id=dc98526f14e84279b002c43ef44cabed&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=b711a007d49d405fa3b1bf14eaa94b81 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:59.510785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c45a3b-f3c5-4e4c-9281-bc822e481bfa None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:59.511338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c45a3b-f3c5-4e4c-9281-bc822e481bfa None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:59.511338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c45a3b-f3c5-4e4c-9281-bc822e481bfa None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:59.513004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 490/990] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:07:59.518893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81daa54-f869-44ff-ad6a-a23e5c081426 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:59.519192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81daa54-f869-44ff-ad6a-a23e5c081426 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:59.519476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81daa54-f869-44ff-ad6a-a23e5c081426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:59.538093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-b81daa54-f869-44ff-ad6a-a23e5c081426 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:07:59.539658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b81daa54-f869-44ff-ad6a-a23e5c081426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:07:59.615731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 501/991] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:07:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:07:59.639476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c45a3b-f3c5-4e4c-9281-bc822e481bfa None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:59.744626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c45a3b-f3c5-4e4c-9281-bc822e481bfa None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:59.771326 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:59.781304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524fd56-12a4-47fd-a842-f6851cece4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:59.782261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524fd56-12a4-47fd-a842-f6851cece4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:59.782647 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524fd56-12a4-47fd-a842-f6851cece4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:59.783026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524fd56-12a4-47fd-a842-f6851cece4f1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:59.783780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524fd56-12a4-47fd-a842-f6851cece4f1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:59.785296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524fd56-12a4-47fd-a842-f6851cece4f1 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-b524fd56-12a4-47fd-a842-f6851cece4f1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:07:59.798516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:07:59.799695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 07:07:59.800899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:07:59.802771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 491/992] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jun 17 07:07:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:07:59.809214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81daa54-f869-44ff-ad6a-a23e5c081426 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:07:59.868380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:07:59.911403 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81daa54-f869-44ff-ad6a-a23e5c081426 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:07:59.945736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b851312-d599-42a4-bfb1-5964c3e7ae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:07:59.946549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b851312-d599-42a4-bfb1-5964c3e7ae72 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:07:59.946905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b851312-d599-42a4-bfb1-5964c3e7ae72 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:07:59.947254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b851312-d599-42a4-bfb1-5964c3e7ae72 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:07:59.948053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b851312-d599-42a4-bfb1-5964c3e7ae72 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:07:59.948460 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b851312-d599-42a4-bfb1-5964c3e7ae72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:07:59.949967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b851312-d599-42a4-bfb1-5964c3e7ae72 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:07:59.956110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 502/993] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jun 17 07:07:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:07:59.962888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524fd56-12a4-47fd-a842-f6851cece4f1 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:00.030549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c5128-a0bf-453f-a279-f6d37ac1d543 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:00.030843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c5128-a0bf-453f-a279-f6d37ac1d543 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:00.031117 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c5128-a0bf-453f-a279-f6d37ac1d543 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:00.032684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 503/994] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:00.038324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dff251-4df7-44d1-9199-20643ed93c35 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:00.038619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dff251-4df7-44d1-9199-20643ed93c35 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:00.038906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dff251-4df7-44d1-9199-20643ed93c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:00.058307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-66dff251-4df7-44d1-9199-20643ed93c35 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:00.059489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-66dff251-4df7-44d1-9199-20643ed93c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:00.070077 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524fd56-12a4-47fd-a842-f6851cece4f1 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:00.106205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcfa7f-ea1d-4930-9902-0630f80af5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:00.106869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcfa7f-ea1d-4930-9902-0630f80af5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:00.107154 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcfa7f-ea1d-4930-9902-0630f80af5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:00.107440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcfa7f-ea1d-4930-9902-0630f80af5c6 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:00.108169 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcfa7f-ea1d-4930-9902-0630f80af5c6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:00.108480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcfa7f-ea1d-4930-9902-0630f80af5c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:00.109856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcfa7f-ea1d-4930-9902-0630f80af5c6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:00.115198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 492/995] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jun 17 07:07:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:00.122216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcfa7f-ea1d-4930-9902-0630f80af5c6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:00.140675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 504/996] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:08:00.165812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dff251-4df7-44d1-9199-20643ed93c35 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:00.187800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:00.220938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:00.240558 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcfa7f-ea1d-4930-9902-0630f80af5c6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:00.269198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dff251-4df7-44d1-9199-20643ed93c35 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:00.278469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b8afa-adf2-46ef-aff7-a689b51a634e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:00.279211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b8afa-adf2-46ef-aff7-a689b51a634e None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:00.279567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b8afa-adf2-46ef-aff7-a689b51a634e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:00.279924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b8afa-adf2-46ef-aff7-a689b51a634e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:00.280784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b8afa-adf2-46ef-aff7-a689b51a634e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:00.282334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b8afa-adf2-46ef-aff7-a689b51a634e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:00.298071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 493/997] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jun 17 07:08:00 2022] POST /identity/v3/endpoints => generated 325 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:08:00.323660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be85d6-5c87-4cfc-847b-2fb2fb02dd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:00.324516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be85d6-5c87-4cfc-847b-2fb2fb02dd73 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:00.324800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be85d6-5c87-4cfc-847b-2fb2fb02dd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:00.325077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be85d6-5c87-4cfc-847b-2fb2fb02dd73 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:00.325744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be85d6-5c87-4cfc-847b-2fb2fb02dd73 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:00.327229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be85d6-5c87-4cfc-847b-2fb2fb02dd73 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-08be85d6-5c87-4cfc-847b-2fb2fb02dd73 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:00.340698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:00.341825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:00.347957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 505/998] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:08:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:00.350331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b8afa-adf2-46ef-aff7-a689b51a634e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:00.457499 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b8afa-adf2-46ef-aff7-a689b51a634e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:00.472747 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:00.496440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cc102-43f1-4679-9201-8530954c4a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:00.497383 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cc102-43f1-4679-9201-8530954c4a97 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:00.497670 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cc102-43f1-4679-9201-8530954c4a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:00.497953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cc102-43f1-4679-9201-8530954c4a97 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:00.498666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cc102-43f1-4679-9201-8530954c4a97 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:00.499006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cc102-43f1-4679-9201-8530954c4a97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:00.500434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cc102-43f1-4679-9201-8530954c4a97 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:00.505328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 494/999] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:08:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:00.511677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be85d6-5c87-4cfc-847b-2fb2fb02dd73 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:00.619573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be85d6-5c87-4cfc-847b-2fb2fb02dd73 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:00.632221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:00.654770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1cd9b-2d07-46cd-b35e-0bb2d50f6f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:00.655444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1cd9b-2d07-46cd-b35e-0bb2d50f6f43 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:00.655798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1cd9b-2d07-46cd-b35e-0bb2d50f6f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:00.656165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1cd9b-2d07-46cd-b35e-0bb2d50f6f43 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:00.656940 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1cd9b-2d07-46cd-b35e-0bb2d50f6f43 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:00.657318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1cd9b-2d07-46cd-b35e-0bb2d50f6f43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:00.658745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1cd9b-2d07-46cd-b35e-0bb2d50f6f43 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:00.663971 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 506/1000] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:08:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:00.669547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cc102-43f1-4679-9201-8530954c4a97 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:00.767339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cc102-43f1-4679-9201-8530954c4a97 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:00.800854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2e18e-39fc-4a70-bcb1-6b4c0b2d8c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:00.801515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2e18e-39fc-4a70-bcb1-6b4c0b2d8c2a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:00.801798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2e18e-39fc-4a70-bcb1-6b4c0b2d8c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:00.802068 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2e18e-39fc-4a70-bcb1-6b4c0b2d8c2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:00.802847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2e18e-39fc-4a70-bcb1-6b4c0b2d8c2a None admin] RBAC: Adding query filter params (interface=public, service_id=8fc07d350ca44119b764144ae0e57eac, region_id=RegionOne) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:00.803159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2e18e-39fc-4a70-bcb1-6b4c0b2d8c2a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:00.804550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2e18e-39fc-4a70-bcb1-6b4c0b2d8c2a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:00.809831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 495/1001] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jun 17 07:08:00 2022] GET /identity/v3/endpoints?service_id=8fc07d350ca44119b764144ae0e57eac&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:00.815440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1cd9b-2d07-46cd-b35e-0bb2d50f6f43 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:00.913026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1cd9b-2d07-46cd-b35e-0bb2d50f6f43 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:00.945282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a950c-9615-428a-8de9-f76518c585e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:00.946589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a950c-9615-428a-8de9-f76518c585e6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:00.946936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a950c-9615-428a-8de9-f76518c585e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:00.947272 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a950c-9615-428a-8de9-f76518c585e6 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:00.948005 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a950c-9615-428a-8de9-f76518c585e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:00.949444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a950c-9615-428a-8de9-f76518c585e6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:00.961151 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 507/1002] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jun 17 07:08:00 2022] GET /identity/v3/services => generated 1727 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:08:01.628777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50fd12-c24e-4bd7-b6c6-dc22c6a4204e None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:01.629070 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50fd12-c24e-4bd7-b6c6-dc22c6a4204e None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:01.629359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50fd12-c24e-4bd7-b6c6-dc22c6a4204e None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:01.630742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 496/1003] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:01.636263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52fe88-5558-4fd0-a3ec-d13d27580c79 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:01.636564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52fe88-5558-4fd0-a3ec-d13d27580c79 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:01.636850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52fe88-5558-4fd0-a3ec-d13d27580c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:01.654579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-4f52fe88-5558-4fd0-a3ec-d13d27580c79 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:01.655696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-4f52fe88-5558-4fd0-a3ec-d13d27580c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:01.690330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:01.725554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 508/1004] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:01.747923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50fd12-c24e-4bd7-b6c6-dc22c6a4204e None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:01.793341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:01.851491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50fd12-c24e-4bd7-b6c6-dc22c6a4204e None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:01.885380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09c0d4-a661-4338-808a-b93124b78e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:01.886161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09c0d4-a661-4338-808a-b93124b78e2c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:01.886439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09c0d4-a661-4338-808a-b93124b78e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:01.886712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09c0d4-a661-4338-808a-b93124b78e2c None admin] PATH_INFO: `/v3/services/image` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:01.887353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09c0d4-a661-4338-808a-b93124b78e2c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:01.888735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09c0d4-a661-4338-808a-b93124b78e2c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-8f09c0d4-a661-4338-808a-b93124b78e2c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:01.900455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:01.901438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:01.907239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 497/1005] 10.222.0.33 () {56 vars in 1105 bytes} [Fri Jun 17 07:08:01 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:01.910019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52fe88-5558-4fd0-a3ec-d13d27580c79 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:02.011745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52fe88-5558-4fd0-a3ec-d13d27580c79 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:02.045895 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4a45f-f4d8-47f2-be07-f114215bdca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:02.046595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4a45f-f4d8-47f2-be07-f114215bdca9 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:02.046893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4a45f-f4d8-47f2-be07-f114215bdca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:02.047177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4a45f-f4d8-47f2-be07-f114215bdca9 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:02.047901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4a45f-f4d8-47f2-be07-f114215bdca9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:02.048244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4a45f-f4d8-47f2-be07-f114215bdca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:02.049771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4a45f-f4d8-47f2-be07-f114215bdca9 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:02.060211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 509/1006] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:02.061685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09c0d4-a661-4338-808a-b93124b78e2c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:02.164488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09c0d4-a661-4338-808a-b93124b78e2c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:02.171428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:02.201203 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b9696-315f-443b-8ecb-ee8a2f014eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:02.201868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b9696-315f-443b-8ecb-ee8a2f014eac None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:02.202148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b9696-315f-443b-8ecb-ee8a2f014eac None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:02.202422 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b9696-315f-443b-8ecb-ee8a2f014eac None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:02.203115 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b9696-315f-443b-8ecb-ee8a2f014eac None admin] RBAC: Adding query filter params (type=image) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:02.203427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b9696-315f-443b-8ecb-ee8a2f014eac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:02.204814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b9696-315f-443b-8ecb-ee8a2f014eac None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:02.209484 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 498/1007] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:02.467773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb2d57-80a2-4b78-ba43-14da37d4dceb None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:02.468159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb2d57-80a2-4b78-ba43-14da37d4dceb None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:02.468370 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb2d57-80a2-4b78-ba43-14da37d4dceb None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:02.469892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 510/1008] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:02.474820 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e7f17-7b59-418b-aa53-cc5f432eba44 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:02.475113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e7f17-7b59-418b-aa53-cc5f432eba44 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:02.475395 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e7f17-7b59-418b-aa53-cc5f432eba44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:02.493249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-072e7f17-7b59-418b-aa53-cc5f432eba44 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:02.494429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-072e7f17-7b59-418b-aa53-cc5f432eba44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:02.561154 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 499/1009] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:02 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:02.584204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb2d57-80a2-4b78-ba43-14da37d4dceb None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:02.619820 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:02.689589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb2d57-80a2-4b78-ba43-14da37d4dceb None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:02.722668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534b088-7ac3-414c-a5e5-916bed554114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:02.723439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534b088-7ac3-414c-a5e5-916bed554114 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:02.723722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534b088-7ac3-414c-a5e5-916bed554114 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:02.723995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534b088-7ac3-414c-a5e5-916bed554114 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:02.725223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534b088-7ac3-414c-a5e5-916bed554114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:02.726714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534b088-7ac3-414c-a5e5-916bed554114 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:02.733550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 511/1010] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:02.735095 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e7f17-7b59-418b-aa53-cc5f432eba44 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:02.829070 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:02.832917 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e7f17-7b59-418b-aa53-cc5f432eba44 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:02.867108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb9c61-1ce4-47b5-8d23-5d5c86f6958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:02.868323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb9c61-1ce4-47b5-8d23-5d5c86f6958d None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:02.868607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb9c61-1ce4-47b5-8d23-5d5c86f6958d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:02.868880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb9c61-1ce4-47b5-8d23-5d5c86f6958d None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:02.869570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb9c61-1ce4-47b5-8d23-5d5c86f6958d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:02.870991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb9c61-1ce4-47b5-8d23-5d5c86f6958d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:02.904101 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 500/1011] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:08:02 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:08:03.098220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30623f0-5a10-4321-b377-1caa1b029716 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:03.098707 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30623f0-5a10-4321-b377-1caa1b029716 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:03.098707 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30623f0-5a10-4321-b377-1caa1b029716 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:03.100097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 512/1012] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:03.105451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dba9c-0f80-4592-882e-7e4e24395f50 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:03.105746 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dba9c-0f80-4592-882e-7e4e24395f50 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:03.106023 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dba9c-0f80-4592-882e-7e4e24395f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:03.123430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-cc1dba9c-0f80-4592-882e-7e4e24395f50 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:03.124639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cc1dba9c-0f80-4592-882e-7e4e24395f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:03.190897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 501/1013] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:03 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:03.214137 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30623f0-5a10-4321-b377-1caa1b029716 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:03.316536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30623f0-5a10-4321-b377-1caa1b029716 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:03.318075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:03.353007 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311005fd-652b-4ca9-8642-cd1debe74fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:03.353717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311005fd-652b-4ca9-8642-cd1debe74fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:03.354001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311005fd-652b-4ca9-8642-cd1debe74fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:03.354277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311005fd-652b-4ca9-8642-cd1debe74fd9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:03.354917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311005fd-652b-4ca9-8642-cd1debe74fd9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:03.356300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311005fd-652b-4ca9-8642-cd1debe74fd9 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-311005fd-652b-4ca9-8642-cd1debe74fd9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:03.368057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:03.369223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:03.370786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 513/1014] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:08:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:03.376121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1dba9c-0f80-4592-882e-7e4e24395f50 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:03.465354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:03.483937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1dba9c-0f80-4592-882e-7e4e24395f50 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:03.516896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a46fc6-1741-49d1-81f4-e20470242c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:03.517552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a46fc6-1741-49d1-81f4-e20470242c8d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:03.517834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a46fc6-1741-49d1-81f4-e20470242c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:03.518109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a46fc6-1741-49d1-81f4-e20470242c8d None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:03.518804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a46fc6-1741-49d1-81f4-e20470242c8d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:03.519163 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a46fc6-1741-49d1-81f4-e20470242c8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:03.520566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a46fc6-1741-49d1-81f4-e20470242c8d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:03.527111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 502/1015] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:08:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:03.531873 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311005fd-652b-4ca9-8642-cd1debe74fd9 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:03.630762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311005fd-652b-4ca9-8642-cd1debe74fd9 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:03.664991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20343422-b4cb-4a65-a156-5d609166ea2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:03.665660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20343422-b4cb-4a65-a156-5d609166ea2b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:03.665945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20343422-b4cb-4a65-a156-5d609166ea2b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:03.666219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20343422-b4cb-4a65-a156-5d609166ea2b None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:03.666920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20343422-b4cb-4a65-a156-5d609166ea2b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:03.667236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20343422-b4cb-4a65-a156-5d609166ea2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:03.668628 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20343422-b4cb-4a65-a156-5d609166ea2b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:03.678201 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 514/1016] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jun 17 07:08:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:03.679722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a46fc6-1741-49d1-81f4-e20470242c8d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:03.779766 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a46fc6-1741-49d1-81f4-e20470242c8d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:03.813106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7f83e-9ab7-4019-9fb7-0ef5770564f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:03.813769 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7f83e-9ab7-4019-9fb7-0ef5770564f8 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:03.814054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7f83e-9ab7-4019-9fb7-0ef5770564f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:03.814334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7f83e-9ab7-4019-9fb7-0ef5770564f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:03.815075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7f83e-9ab7-4019-9fb7-0ef5770564f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:03.816468 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7f83e-9ab7-4019-9fb7-0ef5770564f8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:03.840104 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 503/1017] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jun 17 07:08:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:08:04.323901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe2398-b40e-4c51-a582-10a722b4a5a2 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:04.324465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe2398-b40e-4c51-a582-10a722b4a5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:04.324465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe2398-b40e-4c51-a582-10a722b4a5a2 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:04.325978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 515/1018] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:04.330845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332b2d-2ce5-4805-bea5-0914d75530f2 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:04.331134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332b2d-2ce5-4805-bea5-0914d75530f2 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:04.331416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332b2d-2ce5-4805-bea5-0914d75530f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:04.336833 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:04.349734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-39332b2d-2ce5-4805-bea5-0914d75530f2 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:04.350909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-39332b2d-2ce5-4805-bea5-0914d75530f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:04.432134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 504/1019] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:04 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:04.449935 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe2398-b40e-4c51-a582-10a722b4a5a2 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:04.482664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:04.561199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe2398-b40e-4c51-a582-10a722b4a5a2 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:04.595705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48569a32-ebbe-4976-9ced-3c2d9f6db8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:04.596318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48569a32-ebbe-4976-9ced-3c2d9f6db8e4 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:04.596610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48569a32-ebbe-4976-9ced-3c2d9f6db8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:04.596893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48569a32-ebbe-4976-9ced-3c2d9f6db8e4 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:04.597626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48569a32-ebbe-4976-9ced-3c2d9f6db8e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:04.599055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48569a32-ebbe-4976-9ced-3c2d9f6db8e4 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:04.620097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 516/1020] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:08:04 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:08:05.066217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589026c-8ab9-44e6-87e2-bdc246ee5719 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:05.066725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589026c-8ab9-44e6-87e2-bdc246ee5719 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:05.066803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589026c-8ab9-44e6-87e2-bdc246ee5719 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:05.068277 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 505/1021] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:05.073741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37b62f-283f-44de-8323-4f8cd4f51392 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:05.074033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37b62f-283f-44de-8323-4f8cd4f51392 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:05.074314 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37b62f-283f-44de-8323-4f8cd4f51392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:05.091627 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3d37b62f-283f-44de-8323-4f8cd4f51392 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:05.092832 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3d37b62f-283f-44de-8323-4f8cd4f51392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:05.177221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 517/1022] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:05.197532 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589026c-8ab9-44e6-87e2-bdc246ee5719 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:05.297823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589026c-8ab9-44e6-87e2-bdc246ee5719 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:05.323176 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:05.331899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f734f5e-bdf2-43b9-9fa6-e75cb05c2610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:05.332732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f734f5e-bdf2-43b9-9fa6-e75cb05c2610 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:05.333016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f734f5e-bdf2-43b9-9fa6-e75cb05c2610 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:05.333293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f734f5e-bdf2-43b9-9fa6-e75cb05c2610 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:05.334536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f734f5e-bdf2-43b9-9fa6-e75cb05c2610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:05.335964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f734f5e-bdf2-43b9-9fa6-e75cb05c2610 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:05.340412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 506/1023] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:05.343864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37b62f-283f-44de-8323-4f8cd4f51392 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:05.362396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:05.450378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37b62f-283f-44de-8323-4f8cd4f51392 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:05.483911 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f60f4-1279-4091-937c-a14bc2f23834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:05.484809 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f60f4-1279-4091-937c-a14bc2f23834 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:05.485241 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f60f4-1279-4091-937c-a14bc2f23834 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:05.485662 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f60f4-1279-4091-937c-a14bc2f23834 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:05.486509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f60f4-1279-4091-937c-a14bc2f23834 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:05.488094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f60f4-1279-4091-937c-a14bc2f23834 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:05.512123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 518/1024] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:08:05 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:08:06.058240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5da6b-f68c-4fc8-8d91-3ede575a60a5 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:06.058530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5da6b-f68c-4fc8-8d91-3ede575a60a5 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:06.058814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5da6b-f68c-4fc8-8d91-3ede575a60a5 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:06.060221 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 507/1025] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:06.065619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34aad3-3789-44b9-9517-d467f7eabe1e None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:06.065909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34aad3-3789-44b9-9517-d467f7eabe1e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:06.066184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34aad3-3789-44b9-9517-d467f7eabe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:06.083020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-fb34aad3-3789-44b9-9517-d467f7eabe1e None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:06.084211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-fb34aad3-3789-44b9-9517-d467f7eabe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:06.148686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 519/1026] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:06.172154 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5da6b-f68c-4fc8-8d91-3ede575a60a5 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:06.272620 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5da6b-f68c-4fc8-8d91-3ede575a60a5 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:06.306436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb75f3c-ae28-453d-a17a-04713c40e2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:06.307242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb75f3c-ae28-453d-a17a-04713c40e2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:06.307527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb75f3c-ae28-453d-a17a-04713c40e2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:06.307798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb75f3c-ae28-453d-a17a-04713c40e2e0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:06.317932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb75f3c-ae28-453d-a17a-04713c40e2e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-3fb75f3c-ae28-453d-a17a-04713c40e2e0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:08:06.326511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:06.327523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:06.328387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:06.328387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:06.328387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:06.328387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:06.328387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:08:06.328387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:08:06.328387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:08:06.328387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:08:06.328387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:08:06.328387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:08:06.328387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 07:08:06.328387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:06.329588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 508/1027] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jun 17 07:08:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:06.333592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34aad3-3789-44b9-9517-d467f7eabe1e None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:06.335875 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:06.443335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34aad3-3789-44b9-9517-d467f7eabe1e None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:06.476694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c9a6d-ad1f-45b7-8fe3-2182c8f8b1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:06.477428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c9a6d-ad1f-45b7-8fe3-2182c8f8b1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:06.477712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c9a6d-ad1f-45b7-8fe3-2182c8f8b1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:06.477985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c9a6d-ad1f-45b7-8fe3-2182c8f8b1e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:06.478717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c9a6d-ad1f-45b7-8fe3-2182c8f8b1e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:06.479021 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c9a6d-ad1f-45b7-8fe3-2182c8f8b1e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:06.480400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c9a6d-ad1f-45b7-8fe3-2182c8f8b1e8 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:06.488602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 520/1028] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jun 17 07:08:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:06.493984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb75f3c-ae28-453d-a17a-04713c40e2e0 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:06.549251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:06.594252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb75f3c-ae28-453d-a17a-04713c40e2e0 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:06.627073 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db878b-8cf6-478e-8eca-6bc6be6ac086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:06.627870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db878b-8cf6-478e-8eca-6bc6be6ac086 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:06.628178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db878b-8cf6-478e-8eca-6bc6be6ac086 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:06.628457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db878b-8cf6-478e-8eca-6bc6be6ac086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:06.638946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db878b-8cf6-478e-8eca-6bc6be6ac086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:06.640413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db878b-8cf6-478e-8eca-6bc6be6ac086 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-c2db878b-8cf6-478e-8eca-6bc6be6ac086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:08:06.657936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:06.658900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:06.659846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:08:06.659846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:08:06.659846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:06.659846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:06.660769 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 509/1029] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:06.665098 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c9a6d-ad1f-45b7-8fe3-2182c8f8b1e8 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:06.735081 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:06.766898 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16478da-be22-4dbd-a63b-e82a470a33c0 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:06.767252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16478da-be22-4dbd-a63b-e82a470a33c0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:06.767476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16478da-be22-4dbd-a63b-e82a470a33c0 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:06.768887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 510/1030] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:08:06.773708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbff5b-b95d-4641-be05-82ffdee8935a None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:06.774046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbff5b-b95d-4641-be05-82ffdee8935a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:06.774327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbff5b-b95d-4641-be05-82ffdee8935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:06.779664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c9a6d-ad1f-45b7-8fe3-2182c8f8b1e8 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:06.791024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-44fbff5b-b95d-4641-be05-82ffdee8935a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:06.792226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-44fbff5b-b95d-4641-be05-82ffdee8935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:06.815322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1b0e8-1a3e-4673-9ab8-1450237eaddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:06.815983 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1b0e8-1a3e-4673-9ab8-1450237eaddf None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:06.816309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1b0e8-1a3e-4673-9ab8-1450237eaddf None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:06.816596 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1b0e8-1a3e-4673-9ab8-1450237eaddf None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:06.817319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1b0e8-1a3e-4673-9ab8-1450237eaddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:06.817631 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1b0e8-1a3e-4673-9ab8-1450237eaddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:06.818998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1b0e8-1a3e-4673-9ab8-1450237eaddf None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:06.830841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 521/1031] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:06.837066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1b0e8-1a3e-4673-9ab8-1450237eaddf None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:06.860367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 511/1032] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:08:06.883102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbff5b-b95d-4641-be05-82ffdee8935a None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:06.926642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:06.947691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1b0e8-1a3e-4673-9ab8-1450237eaddf None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:06.974212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:06.985564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2b459-7d8d-4379-9074-99624610dd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:06.986409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2b459-7d8d-4379-9074-99624610dd80 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:06.986688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2b459-7d8d-4379-9074-99624610dd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:06.986962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2b459-7d8d-4379-9074-99624610dd80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:06.987962 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbff5b-b95d-4641-be05-82ffdee8935a None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:06.994006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2b459-7d8d-4379-9074-99624610dd80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:06.995429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2b459-7d8d-4379-9074-99624610dd80 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-93e2b459-7d8d-4379-9074-99624610dd80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:08:07.009913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:07.010871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:07.017198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:07.017198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:07.017198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:08:07.017198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:08:07.017198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:08:07.017198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:08:07.017198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:08:07.017198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:08:07.017198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:08:07.017198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:07.017198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 522/1033] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jun 17 07:08:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:07.017198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2b459-7d8d-4379-9074-99624610dd80 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:07.032859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd952d31-3690-4bcc-adfb-944c3682f17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:07.033651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd952d31-3690-4bcc-adfb-944c3682f17f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:07.033936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd952d31-3690-4bcc-adfb-944c3682f17f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:07.034210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd952d31-3690-4bcc-adfb-944c3682f17f None admin] PATH_INFO: `/v3/services/image` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:07.034858 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd952d31-3690-4bcc-adfb-944c3682f17f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:07.036243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd952d31-3690-4bcc-adfb-944c3682f17f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-dd952d31-3690-4bcc-adfb-944c3682f17f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:07.047515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:07.048557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:07.049414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 512/1034] 10.222.0.33 () {56 vars in 1105 bytes} [Fri Jun 17 07:08:06 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:07.054321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd952d31-3690-4bcc-adfb-944c3682f17f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:07.125103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2b459-7d8d-4379-9074-99624610dd80 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:07.153471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd952d31-3690-4bcc-adfb-944c3682f17f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:07.155423 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:07.160093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db24e22-f044-4864-acf5-3f1571dfa620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:07.160757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db24e22-f044-4864-acf5-3f1571dfa620 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:07.161042 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db24e22-f044-4864-acf5-3f1571dfa620 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:07.161319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db24e22-f044-4864-acf5-3f1571dfa620 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:07.162158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db24e22-f044-4864-acf5-3f1571dfa620 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:07.162705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db24e22-f044-4864-acf5-3f1571dfa620 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:07.163885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db24e22-f044-4864-acf5-3f1571dfa620 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:07.185223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 523/1035] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jun 17 07:08:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:07.190599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db24e22-f044-4864-acf5-3f1571dfa620 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:07.197276 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99228fb-fd61-4e38-acbf-a231a3c6f4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:07.197947 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99228fb-fd61-4e38-acbf-a231a3c6f4ab None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:07.198229 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99228fb-fd61-4e38-acbf-a231a3c6f4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:07.198497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99228fb-fd61-4e38-acbf-a231a3c6f4ab None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:07.199205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99228fb-fd61-4e38-acbf-a231a3c6f4ab None admin] RBAC: Adding query filter params (name=image) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:07.199517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99228fb-fd61-4e38-acbf-a231a3c6f4ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:07.200918 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99228fb-fd61-4e38-acbf-a231a3c6f4ab None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:07.206158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 513/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:07.211660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99228fb-fd61-4e38-acbf-a231a3c6f4ab None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:07.240671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:07.299892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db24e22-f044-4864-acf5-3f1571dfa620 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:07.316255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99228fb-fd61-4e38-acbf-a231a3c6f4ab None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:07.335605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c4b7a-db2a-4d1e-b8dd-07d91fcd911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:07.336865 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c4b7a-db2a-4d1e-b8dd-07d91fcd911d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:07.337189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c4b7a-db2a-4d1e-b8dd-07d91fcd911d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:07.337469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c4b7a-db2a-4d1e-b8dd-07d91fcd911d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:07.353111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e6671-6e04-44a3-8bb1-0b8729b91e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:07.354026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e6671-6e04-44a3-8bb1-0b8729b91e97 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:07.354475 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e6671-6e04-44a3-8bb1-0b8729b91e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:07.355029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e6671-6e04-44a3-8bb1-0b8729b91e97 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:07.356096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e6671-6e04-44a3-8bb1-0b8729b91e97 None admin] RBAC: Adding query filter params (type=image) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:07.356576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e6671-6e04-44a3-8bb1-0b8729b91e97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:07.358193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e6671-6e04-44a3-8bb1-0b8729b91e97 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:07.363951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 514/1037] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:07.369827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e6671-6e04-44a3-8bb1-0b8729b91e97 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:07.404544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:07.419049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c4b7a-db2a-4d1e-b8dd-07d91fcd911d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:07.420618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c4b7a-db2a-4d1e-b8dd-07d91fcd911d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:07.464862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:07.481953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e6671-6e04-44a3-8bb1-0b8729b91e97 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:07.516559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b9c6a-cce4-4330-b5fe-6525b19911e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:07.517220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b9c6a-cce4-4330-b5fe-6525b19911e4 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:07.517508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b9c6a-cce4-4330-b5fe-6525b19911e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:07.517791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b9c6a-cce4-4330-b5fe-6525b19911e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:07.518572 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b9c6a-cce4-4330-b5fe-6525b19911e4 None admin] RBAC: Adding query filter params (interface=public, service_id=e472a6f190dd4c2e9a81245e43bfe846, region_id=RegionOne) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:07.518888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b9c6a-cce4-4330-b5fe-6525b19911e4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:07.520271 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b9c6a-cce4-4330-b5fe-6525b19911e4 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:07.526391 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 515/1038] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jun 17 07:08:07 2022] GET /identity/v3/endpoints?service_id=e472a6f190dd4c2e9a81245e43bfe846&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:07.528690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 524/1039] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:08:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:08:07.533984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b9c6a-cce4-4330-b5fe-6525b19911e4 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:07.534119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c4b7a-db2a-4d1e-b8dd-07d91fcd911d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:07.638822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:07.641784 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c4b7a-db2a-4d1e-b8dd-07d91fcd911d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:07.648391 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b9c6a-cce4-4330-b5fe-6525b19911e4 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:07.682114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d60e1-7cce-46f6-90a0-4a9bdfa2c84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:07.682945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d60e1-7cce-46f6-90a0-4a9bdfa2c84c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:07.683229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d60e1-7cce-46f6-90a0-4a9bdfa2c84c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:07.683506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d60e1-7cce-46f6-90a0-4a9bdfa2c84c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:07.694779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d60e1-7cce-46f6-90a0-4a9bdfa2c84c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:07.696790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d60e1-7cce-46f6-90a0-4a9bdfa2c84c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:07.698109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50024f-f6b9-43c7-a6ae-da3fcbdb2337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:07.698792 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:07.700268 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50024f-f6b9-43c7-a6ae-da3fcbdb2337 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:07.700762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50024f-f6b9-43c7-a6ae-da3fcbdb2337 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:07.701449 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50024f-f6b9-43c7-a6ae-da3fcbdb2337 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:07.702457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50024f-f6b9-43c7-a6ae-da3fcbdb2337 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:07.704434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50024f-f6b9-43c7-a6ae-da3fcbdb2337 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:07.720556 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 516/1040] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jun 17 07:08:07 2022] GET /identity/v3/services => generated 1957 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-350d60e1-7cce-46f6-90a0-4a9bdfa2c84c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:08:07.720791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:07.721818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:07.722929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:08:07.722929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:08:07.722929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:07.722929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:07.722929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 525/1041] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:07.723213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c71df6-76d4-411c-87c2-887fa2629048 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:07.723213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c71df6-76d4-411c-87c2-887fa2629048 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:07.723213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c71df6-76d4-411c-87c2-887fa2629048 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:07.723832 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d60e1-7cce-46f6-90a0-4a9bdfa2c84c None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:07.726675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 517/1042] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:07.729796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f795274-65b9-4710-aa0b-a94e93b137d8 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:07.730088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f795274-65b9-4710-aa0b-a94e93b137d8 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:07.730369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f795274-65b9-4710-aa0b-a94e93b137d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:07.747629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-7f795274-65b9-4710-aa0b-a94e93b137d8 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:07.748901 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7f795274-65b9-4710-aa0b-a94e93b137d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:07.828095 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 518/1043] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:08:07.842348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d60e1-7cce-46f6-90a0-4a9bdfa2c84c None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:07.851614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f795274-65b9-4710-aa0b-a94e93b137d8 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:07.882911 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a86f4-b957-4ace-86c8-c8f4d74e6a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:07.883611 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a86f4-b957-4ace-86c8-c8f4d74e6a5b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:07.883895 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a86f4-b957-4ace-86c8-c8f4d74e6a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:07.884227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a86f4-b957-4ace-86c8-c8f4d74e6a5b None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:07.884949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a86f4-b957-4ace-86c8-c8f4d74e6a5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:07.885258 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a86f4-b957-4ace-86c8-c8f4d74e6a5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:07.886636 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a86f4-b957-4ace-86c8-c8f4d74e6a5b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:07.899271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 526/1044] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:07.905139 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a86f4-b957-4ace-86c8-c8f4d74e6a5b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:07.948391 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:07.962168 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f795274-65b9-4710-aa0b-a94e93b137d8 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:07.962970 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:08.000963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deddc29-35be-4de6-8d7e-f2dc498f81e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:08.001771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deddc29-35be-4de6-8d7e-f2dc498f81e9 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:08.002243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deddc29-35be-4de6-8d7e-f2dc498f81e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:08.002526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deddc29-35be-4de6-8d7e-f2dc498f81e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:08.012905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a86f4-b957-4ace-86c8-c8f4d74e6a5b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:08.013930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deddc29-35be-4de6-8d7e-f2dc498f81e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-5deddc29-35be-4de6-8d7e-f2dc498f81e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:08:08.021646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:08.022785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:08.023712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 519/1045] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:08:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:08.029781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deddc29-35be-4de6-8d7e-f2dc498f81e9 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:08.057321 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de24722-de44-45a7-884c-7f694821f77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:08.058220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24722-de44-45a7-884c-7f694821f77e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:08.058569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24722-de44-45a7-884c-7f694821f77e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:08.058995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24722-de44-45a7-884c-7f694821f77e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:08.066483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de24722-de44-45a7-884c-7f694821f77e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:08.067449 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de24722-de44-45a7-884c-7f694821f77e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-3de24722-de44-45a7-884c-7f694821f77e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:08:08.082381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:08.083475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:08.089813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:08:08.089813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:08:08.089813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:08:08.089813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:08:08.089813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:08:08.089813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:08:08.089813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:08:08.089813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:08.089813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 527/1046] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:08:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:08.089813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de24722-de44-45a7-884c-7f694821f77e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:08.142491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deddc29-35be-4de6-8d7e-f2dc498f81e9 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:08.179622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128ab60-fdc9-4062-b097-4c9301982420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:08.180855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128ab60-fdc9-4062-b097-4c9301982420 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:08.181145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128ab60-fdc9-4062-b097-4c9301982420 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:08.181425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128ab60-fdc9-4062-b097-4c9301982420 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:08.182172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128ab60-fdc9-4062-b097-4c9301982420 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:08.182593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128ab60-fdc9-4062-b097-4c9301982420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:08.184101 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128ab60-fdc9-4062-b097-4c9301982420 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:08.189333 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de24722-de44-45a7-884c-7f694821f77e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:08.193159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 520/1047] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:08:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:08.198958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128ab60-fdc9-4062-b097-4c9301982420 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:08.207477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:08.211893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:08.234673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ee46b-71ff-4e0e-b66b-474f05b73e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:08.235336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ee46b-71ff-4e0e-b66b-474f05b73e38 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:08.235624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ee46b-71ff-4e0e-b66b-474f05b73e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:08.235903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ee46b-71ff-4e0e-b66b-474f05b73e38 None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:08.236723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ee46b-71ff-4e0e-b66b-474f05b73e38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:08.237039 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ee46b-71ff-4e0e-b66b-474f05b73e38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:08.238463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ee46b-71ff-4e0e-b66b-474f05b73e38 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:08.250862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 528/1048] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:08:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:08.256398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ee46b-71ff-4e0e-b66b-474f05b73e38 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:08.304518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128ab60-fdc9-4062-b097-4c9301982420 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:08.340672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e85c22-8ad7-4d13-bf23-aa165b1ed86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:08.341820 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e85c22-8ad7-4d13-bf23-aa165b1ed86c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:08.342343 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e85c22-8ad7-4d13-bf23-aa165b1ed86c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:08.342801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e85c22-8ad7-4d13-bf23-aa165b1ed86c None admin] PATH_INFO: `/v3/users/5b8d9ae7d08341e38d1b0477c6eb2eb5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:08.356525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e85c22-8ad7-4d13-bf23-aa165b1ed86c None admin] RBAC: Authorizing `identity:get_user(user_id=5b8d9ae7d08341e38d1b0477c6eb2eb5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:08.358483 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e85c22-8ad7-4d13-bf23-aa165b1ed86c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:08.360723 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 521/1049] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:08 2022] GET /identity/v3/users/5b8d9ae7d08341e38d1b0477c6eb2eb5 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:08.360920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ee46b-71ff-4e0e-b66b-474f05b73e38 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:08.366853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e85c22-8ad7-4d13-bf23-aa165b1ed86c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:08.401546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efadfb-1495-4065-a974-0c5ff3fea5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:08.402465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efadfb-1495-4065-a974-0c5ff3fea5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:08.402928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efadfb-1495-4065-a974-0c5ff3fea5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:08.403359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efadfb-1495-4065-a974-0c5ff3fea5a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:08.404383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efadfb-1495-4065-a974-0c5ff3fea5a9 None admin] RBAC: Adding query filter params (role.id=b711a007d49d405fa3b1bf14eaa94b81, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=8d3baa16824f40f3a4c783e210b262f4) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:08.404845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efadfb-1495-4065-a974-0c5ff3fea5a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:08.406459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efadfb-1495-4065-a974-0c5ff3fea5a9 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:08.413022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 529/1050] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:08:08 2022] GET /identity/v3/role_assignments?user.id=8d3baa16824f40f3a4c783e210b262f4&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=b711a007d49d405fa3b1bf14eaa94b81 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:08.455911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:08.492895 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e85c22-8ad7-4d13-bf23-aa165b1ed86c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:08.530087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462cc9a-38d6-489a-b4e6-b1acb7bd7ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:08.530766 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462cc9a-38d6-489a-b4e6-b1acb7bd7ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:08.531053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462cc9a-38d6-489a-b4e6-b1acb7bd7ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:08.531333 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462cc9a-38d6-489a-b4e6-b1acb7bd7ab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:08.603729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462cc9a-38d6-489a-b4e6-b1acb7bd7ab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:08.605241 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462cc9a-38d6-489a-b4e6-b1acb7bd7ab2 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:08.669797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:08.709625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 522/1051] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:08:08 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 07:08:08.710743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efadfb-1495-4065-a974-0c5ff3fea5a9 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:08.752665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:08.814629 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efadfb-1495-4065-a974-0c5ff3fea5a9 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:08.851373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0d42b-426d-471b-8e1b-40099f524d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:08.852297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0d42b-426d-471b-8e1b-40099f524d51 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:08.852584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0d42b-426d-471b-8e1b-40099f524d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:08.852861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0d42b-426d-471b-8e1b-40099f524d51 None admin] PATH_INFO: `/v3/projects/fa1e8038919344b08141ccb5518af99b` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:08.854066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0d42b-426d-471b-8e1b-40099f524d51 None admin] RBAC: Authorizing `identity:get_project(project_id=fa1e8038919344b08141ccb5518af99b)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:08.855506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0d42b-426d-471b-8e1b-40099f524d51 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:08.857398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 530/1052] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:08:08 2022] GET /identity/v3/projects/fa1e8038919344b08141ccb5518af99b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:08.863209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462cc9a-38d6-489a-b4e6-b1acb7bd7ab2 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:08.965239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462cc9a-38d6-489a-b4e6-b1acb7bd7ab2 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:09.000268 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c983d-5959-487c-bce5-44cafaa2c412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:09.000919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c983d-5959-487c-bce5-44cafaa2c412 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:09.001214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c983d-5959-487c-bce5-44cafaa2c412 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:09.001490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c983d-5959-487c-bce5-44cafaa2c412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:09.002318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c983d-5959-487c-bce5-44cafaa2c412 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=fa1e8038919344b08141ccb5518af99b, user.id=5b8d9ae7d08341e38d1b0477c6eb2eb5) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:09.002632 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c983d-5959-487c-bce5-44cafaa2c412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:09.004083 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c983d-5959-487c-bce5-44cafaa2c412 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:09.016331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 523/1053] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:08:08 2022] GET /identity/v3/role_assignments?user.id=5b8d9ae7d08341e38d1b0477c6eb2eb5&scope.project.id=fa1e8038919344b08141ccb5518af99b&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:09.884957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b17a0-c27c-4108-9602-4ac7cd229da5 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:09.885498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b17a0-c27c-4108-9602-4ac7cd229da5 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:09.885498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b17a0-c27c-4108-9602-4ac7cd229da5 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:09.887148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 531/1054] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:09.894190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a46fbe-15a8-47cd-91c8-8d942014179f None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:09.894190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a46fbe-15a8-47cd-91c8-8d942014179f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:09.894190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a46fbe-15a8-47cd-91c8-8d942014179f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:09.910866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-d4a46fbe-15a8-47cd-91c8-8d942014179f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:09.912049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d4a46fbe-15a8-47cd-91c8-8d942014179f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:09.940030 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:09.983657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 524/1055] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:10.006412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b17a0-c27c-4108-9602-4ac7cd229da5 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:10.117684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b17a0-c27c-4108-9602-4ac7cd229da5 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:10.125395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:10.154008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a5da2-28eb-4750-8d97-d2353c5e281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:10.154718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a5da2-28eb-4750-8d97-d2353c5e281b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:10.154997 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a5da2-28eb-4750-8d97-d2353c5e281b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:10.155267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a5da2-28eb-4750-8d97-d2353c5e281b None admin] PATH_INFO: `/v3/services/image` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:10.155966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a5da2-28eb-4750-8d97-d2353c5e281b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:10.157485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a5da2-28eb-4750-8d97-d2353c5e281b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-948a5da2-28eb-4750-8d97-d2353c5e281b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:10.170200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:10.171171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 07:08:10.172113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:10.176499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 532/1056] 10.222.0.33 () {56 vars in 1105 bytes} [Fri Jun 17 07:08:10 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:10.177499 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a46fbe-15a8-47cd-91c8-8d942014179f None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:10.277581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a46fbe-15a8-47cd-91c8-8d942014179f None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:10.311707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc62106-2593-44ea-bf3c-c514ed7d36db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:10.312393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc62106-2593-44ea-bf3c-c514ed7d36db None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:10.312678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc62106-2593-44ea-bf3c-c514ed7d36db None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:10.312953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc62106-2593-44ea-bf3c-c514ed7d36db None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:10.313648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc62106-2593-44ea-bf3c-c514ed7d36db None admin] RBAC: Adding query filter params (name=image) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:10.313954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc62106-2593-44ea-bf3c-c514ed7d36db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:10.315309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc62106-2593-44ea-bf3c-c514ed7d36db None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:10.320309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 525/1057] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:10.326516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a5da2-28eb-4750-8d97-d2353c5e281b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:10.431061 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a5da2-28eb-4750-8d97-d2353c5e281b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:10.464850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0bf9-768f-4eed-83e1-3457aa3c1cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:10.465505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0bf9-768f-4eed-83e1-3457aa3c1cae None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:10.465788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0bf9-768f-4eed-83e1-3457aa3c1cae None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:10.466065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0bf9-768f-4eed-83e1-3457aa3c1cae None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:10.466761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef0bf9-768f-4eed-83e1-3457aa3c1cae None admin] RBAC: Adding query filter params (type=image) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:10.467073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef0bf9-768f-4eed-83e1-3457aa3c1cae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:10.468464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef0bf9-768f-4eed-83e1-3457aa3c1cae None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:10.478899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 533/1058] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:10.480652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc62106-2593-44ea-bf3c-c514ed7d36db None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:10.512102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:10.553595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10386396-7350-4f74-bd03-f61bde5cd84a None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:10.553904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10386396-7350-4f74-bd03-f61bde5cd84a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:10.554170 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10386396-7350-4f74-bd03-f61bde5cd84a None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:10.555712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 534/1059] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:08:10.560491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35fdc1-5b2d-44c3-8d47-ed6d7d9680e4 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:10.560790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35fdc1-5b2d-44c3-8d47-ed6d7d9680e4 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:10.561075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35fdc1-5b2d-44c3-8d47-ed6d7d9680e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:10.577670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-7a35fdc1-5b2d-44c3-8d47-ed6d7d9680e4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:10.578863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7a35fdc1-5b2d-44c3-8d47-ed6d7d9680e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:10.590524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc62106-2593-44ea-bf3c-c514ed7d36db None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:10.606229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:10.632762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7ccb4-0a78-4110-a167-f4866ae76dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:10.633513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7ccb4-0a78-4110-a167-f4866ae76dc1 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:10.633804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7ccb4-0a78-4110-a167-f4866ae76dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:10.634077 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7ccb4-0a78-4110-a167-f4866ae76dc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:10.634809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7ccb4-0a78-4110-a167-f4866ae76dc1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:10.636218 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7ccb4-0a78-4110-a167-f4866ae76dc1 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:10.650307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 535/1060] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:08:10.659697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 526/1061] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jun 17 07:08:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:08:10.672890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35fdc1-5b2d-44c3-8d47-ed6d7d9680e4 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:10.781651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35fdc1-5b2d-44c3-8d47-ed6d7d9680e4 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:10.825390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34b3a4-3389-40c4-a893-6d8edb0cd7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:10.826263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34b3a4-3389-40c4-a893-6d8edb0cd7ee None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:10.826493 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34b3a4-3389-40c4-a893-6d8edb0cd7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:10.826764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34b3a4-3389-40c4-a893-6d8edb0cd7ee None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:10.837581 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34b3a4-3389-40c4-a893-6d8edb0cd7ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-da34b3a4-3389-40c4-a893-6d8edb0cd7ee None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:08:10.845952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:10.847094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:10.848389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 536/1062] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jun 17 07:08:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:10.853772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7ccb4-0a78-4110-a167-f4866ae76dc1 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:10.942455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:10.962817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7ccb4-0a78-4110-a167-f4866ae76dc1 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:10.998326 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf618db-0054-48bd-ac3a-e703fcc29209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:10.999004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf618db-0054-48bd-ac3a-e703fcc29209 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:10.999295 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf618db-0054-48bd-ac3a-e703fcc29209 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:10.999578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf618db-0054-48bd-ac3a-e703fcc29209 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:11.000354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf618db-0054-48bd-ac3a-e703fcc29209 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:11.000668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf618db-0054-48bd-ac3a-e703fcc29209 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:11.002046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf618db-0054-48bd-ac3a-e703fcc29209 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:11.014942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 527/1063] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jun 17 07:08:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:11.016104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34b3a4-3389-40c4-a893-6d8edb0cd7ee None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:11.040260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:11.118940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1daf79-4987-4872-ad79-45fc68676542 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:11.119239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1daf79-4987-4872-ad79-45fc68676542 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:11.119516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1daf79-4987-4872-ad79-45fc68676542 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:11.124106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 528/1064] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:08:11.125841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65863356-f70f-43fa-8f80-4299fa2dba29 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:11.126130 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65863356-f70f-43fa-8f80-4299fa2dba29 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:11.126408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65863356-f70f-43fa-8f80-4299fa2dba29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:11.127143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34b3a4-3389-40c4-a893-6d8edb0cd7ee None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:11.143261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-65863356-f70f-43fa-8f80-4299fa2dba29 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:11.144483 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-65863356-f70f-43fa-8f80-4299fa2dba29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:11.162834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5200ca-304a-4357-b29d-3067e32b3dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:11.163635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5200ca-304a-4357-b29d-3067e32b3dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:11.163933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5200ca-304a-4357-b29d-3067e32b3dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:11.164792 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5200ca-304a-4357-b29d-3067e32b3dd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:11.176851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5200ca-304a-4357-b29d-3067e32b3dd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:11.178280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5200ca-304a-4357-b29d-3067e32b3dd5 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-fe5200ca-304a-4357-b29d-3067e32b3dd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:08:11.197586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:11.198539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:11.199467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:08:11.199467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:08:11.199467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:11.199467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:11.199467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 537/1065] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:11.204356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5200ca-304a-4357-b29d-3067e32b3dd5 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:11.220225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 529/1066] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:08:11.239756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65863356-f70f-43fa-8f80-4299fa2dba29 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:11.281159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:11.314109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5200ca-304a-4357-b29d-3067e32b3dd5 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:11.314502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:11.341370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65863356-f70f-43fa-8f80-4299fa2dba29 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:11.351081 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac5555-9055-4509-b65e-c0b8477b7a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:11.351820 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac5555-9055-4509-b65e-c0b8477b7a7e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:11.352624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac5555-9055-4509-b65e-c0b8477b7a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:11.352899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac5555-9055-4509-b65e-c0b8477b7a7e None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:11.353650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac5555-9055-4509-b65e-c0b8477b7a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:11.353952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac5555-9055-4509-b65e-c0b8477b7a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:11.355312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac5555-9055-4509-b65e-c0b8477b7a7e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:11.367768 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 538/1067] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:11.372973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac5555-9055-4509-b65e-c0b8477b7a7e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:11.376767 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e135e-6d47-4b35-a9ed-44ef9d50c7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:11.377569 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e135e-6d47-4b35-a9ed-44ef9d50c7cc None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:11.377859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e135e-6d47-4b35-a9ed-44ef9d50c7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:11.378141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e135e-6d47-4b35-a9ed-44ef9d50c7cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:11.388445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e135e-6d47-4b35-a9ed-44ef9d50c7cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-dd2e135e-6d47-4b35-a9ed-44ef9d50c7cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:08:11.396415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:11.397310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:11.398150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 530/1068] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:08:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:11.404489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e135e-6d47-4b35-a9ed-44ef9d50c7cc None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:11.495768 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac5555-9055-4509-b65e-c0b8477b7a7e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:11.507532 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e135e-6d47-4b35-a9ed-44ef9d50c7cc None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:11.527231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:11.532905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5147b83-4bb8-4f53-8f00-d4c88aaaf71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:11.533694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5147b83-4bb8-4f53-8f00-d4c88aaaf71f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:11.533982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5147b83-4bb8-4f53-8f00-d4c88aaaf71f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:11.534261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5147b83-4bb8-4f53-8f00-d4c88aaaf71f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:11.541545 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5147b83-4bb8-4f53-8f00-d4c88aaaf71f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:11.542981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5147b83-4bb8-4f53-8f00-d4c88aaaf71f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:11.543350 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac23cf-b12b-46fa-ac3c-15038bf356b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:11.543979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac23cf-b12b-46fa-ac3c-15038bf356b7 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:11.544302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac23cf-b12b-46fa-ac3c-15038bf356b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:11.544652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac23cf-b12b-46fa-ac3c-15038bf356b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:11.545327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac23cf-b12b-46fa-ac3c-15038bf356b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:11.545950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac23cf-b12b-46fa-ac3c-15038bf356b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:11.546998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac23cf-b12b-46fa-ac3c-15038bf356b7 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:11.548497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:11.557195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 531/1069] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:08:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-f5147b83-4bb8-4f53-8f00-d4c88aaaf71f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:08:11.558109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:11.559040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:11.564927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:11.564927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:11.564927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:08:11.564927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:08:11.564927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:08:11.564927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:08:11.564927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:08:11.564927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:08:11.564927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:08:11.564927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:11.564927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 539/1070] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jun 17 07:08:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:11.565688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac23cf-b12b-46fa-ac3c-15038bf356b7 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:11.565871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5147b83-4bb8-4f53-8f00-d4c88aaaf71f None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:11.665955 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac23cf-b12b-46fa-ac3c-15038bf356b7 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:11.673841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5147b83-4bb8-4f53-8f00-d4c88aaaf71f None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:11.701144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0229a-79aa-4c42-a22e-8f3c6485a951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:11.702012 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0229a-79aa-4c42-a22e-8f3c6485a951 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:11.702295 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0229a-79aa-4c42-a22e-8f3c6485a951 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:11.702575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0229a-79aa-4c42-a22e-8f3c6485a951 None admin] PATH_INFO: `/v3/users/5b8d9ae7d08341e38d1b0477c6eb2eb5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:11.703761 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0229a-79aa-4c42-a22e-8f3c6485a951 None admin] RBAC: Authorizing `identity:get_user(user_id=5b8d9ae7d08341e38d1b0477c6eb2eb5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:11.705225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0229a-79aa-4c42-a22e-8f3c6485a951 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:11.707003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 532/1071] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:11 2022] GET /identity/v3/users/5b8d9ae7d08341e38d1b0477c6eb2eb5 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:11.710805 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb554241-b24a-46e4-a2bf-b864c02505b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:11.712207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb554241-b24a-46e4-a2bf-b864c02505b8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:11.712338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb554241-b24a-46e4-a2bf-b864c02505b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:11.712601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb554241-b24a-46e4-a2bf-b864c02505b8 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:11.713040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0229a-79aa-4c42-a22e-8f3c6485a951 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:11.713429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb554241-b24a-46e4-a2bf-b864c02505b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:11.713780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb554241-b24a-46e4-a2bf-b864c02505b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:11.715156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb554241-b24a-46e4-a2bf-b864c02505b8 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:11.737503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 540/1072] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jun 17 07:08:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:11.738722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb554241-b24a-46e4-a2bf-b864c02505b8 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:11.781990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:11.798699 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:11.823773 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0229a-79aa-4c42-a22e-8f3c6485a951 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:11.856684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb554241-b24a-46e4-a2bf-b864c02505b8 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:11.858960 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cde4f-95c8-4951-a92e-ef091854a2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:11.859837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cde4f-95c8-4951-a92e-ef091854a2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:11.860149 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cde4f-95c8-4951-a92e-ef091854a2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:11.860435 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cde4f-95c8-4951-a92e-ef091854a2a9 None admin] PATH_INFO: `/v3/projects/fa1e8038919344b08141ccb5518af99b` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:11.861639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1cde4f-95c8-4951-a92e-ef091854a2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=fa1e8038919344b08141ccb5518af99b)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:11.863080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1cde4f-95c8-4951-a92e-ef091854a2a9 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:11.864840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 533/1073] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:08:11 2022] GET /identity/v3/projects/fa1e8038919344b08141ccb5518af99b => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:11.871055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cde4f-95c8-4951-a92e-ef091854a2a9 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:11.894768 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a3a96-d65e-423f-a66b-08a95ef3558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:11.895565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a3a96-d65e-423f-a66b-08a95ef3558b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:11.895856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a3a96-d65e-423f-a66b-08a95ef3558b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:11.896165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a3a96-d65e-423f-a66b-08a95ef3558b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:11.907388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a3a96-d65e-423f-a66b-08a95ef3558b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:11.908827 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a3a96-d65e-423f-a66b-08a95ef3558b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-be3a3a96-d65e-423f-a66b-08a95ef3558b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:08:11.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:11.929279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:11.931073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:08:11.931073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:08:11.931073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:11.931073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:11.931073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 541/1074] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:11.935140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a3a96-d65e-423f-a66b-08a95ef3558b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:11.978366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cde4f-95c8-4951-a92e-ef091854a2a9 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:12.018465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:12.024094 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b8dd6-7fc4-4d89-85a8-6a445997aad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:12.025231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b8dd6-7fc4-4d89-85a8-6a445997aad0 None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:12.025518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b8dd6-7fc4-4d89-85a8-6a445997aad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:12.025793 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b8dd6-7fc4-4d89-85a8-6a445997aad0 None admin] PATH_INFO: `/v3/projects/fa1e8038919344b08141ccb5518af99b/users/5b8d9ae7d08341e38d1b0477c6eb2eb5/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:12.027783 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b8dd6-7fc4-4d89-85a8-6a445997aad0 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa1e8038919344b08141ccb5518af99b, user_id=5b8d9ae7d08341e38d1b0477c6eb2eb5, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:12.029286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b8dd6-7fc4-4d89-85a8-6a445997aad0 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:12.041939 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a3a96-d65e-423f-a66b-08a95ef3558b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:12.042493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 534/1075] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:08:11 2022] PUT /identity/v3/projects/fa1e8038919344b08141ccb5518af99b/users/5b8d9ae7d08341e38d1b0477c6eb2eb5/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:08:12.045454 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:12.083795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76eab77-9887-4dff-b1ee-8bd53f39a187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:12.084427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76eab77-9887-4dff-b1ee-8bd53f39a187 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:12.084708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76eab77-9887-4dff-b1ee-8bd53f39a187 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:12.084982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76eab77-9887-4dff-b1ee-8bd53f39a187 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:12.085710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76eab77-9887-4dff-b1ee-8bd53f39a187 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:12.086047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76eab77-9887-4dff-b1ee-8bd53f39a187 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:12.087430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76eab77-9887-4dff-b1ee-8bd53f39a187 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:12.104938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 542/1076] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:12.106338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b8dd6-7fc4-4d89-85a8-6a445997aad0 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:12.219549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b8dd6-7fc4-4d89-85a8-6a445997aad0 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:12.254195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9c7c6-c313-4424-8031-87c4bfb962e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:12.254979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9c7c6-c313-4424-8031-87c4bfb962e1 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:12.255268 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9c7c6-c313-4424-8031-87c4bfb962e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:12.255547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9c7c6-c313-4424-8031-87c4bfb962e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:12.264099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9c7c6-c313-4424-8031-87c4bfb962e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:12.265551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9c7c6-c313-4424-8031-87c4bfb962e1 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-9fd9c7c6-c313-4424-8031-87c4bfb962e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:08:12.279342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:12.280495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:12.281430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:08:12.281430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:08:12.281430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:08:12.281430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:08:12.281430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:08:12.281430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:08:12.281430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:08:12.281430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:12.281430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 535/1077] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:08:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:12.287903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76eab77-9887-4dff-b1ee-8bd53f39a187 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:12.392634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76eab77-9887-4dff-b1ee-8bd53f39a187 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:12.427234 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2228c6-be4e-4a17-9e35-3951fc0f084d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:12.427893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2228c6-be4e-4a17-9e35-3951fc0f084d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:12.428201 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2228c6-be4e-4a17-9e35-3951fc0f084d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:12.428476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2228c6-be4e-4a17-9e35-3951fc0f084d None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:12.429285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2228c6-be4e-4a17-9e35-3951fc0f084d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:12.429594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2228c6-be4e-4a17-9e35-3951fc0f084d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:12.431013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2228c6-be4e-4a17-9e35-3951fc0f084d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:12.443207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 543/1078] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:08:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:12.449318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9c7c6-c313-4424-8031-87c4bfb962e1 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:12.503810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:12.561494 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9c7c6-c313-4424-8031-87c4bfb962e1 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:12.594450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac6464-9416-48aa-b902-481b40253ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:12.595332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac6464-9416-48aa-b902-481b40253ab8 None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:12.595612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac6464-9416-48aa-b902-481b40253ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:12.595887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac6464-9416-48aa-b902-481b40253ab8 None admin] PATH_INFO: `/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/8d3baa16824f40f3a4c783e210b262f4/roles/b711a007d49d405fa3b1bf14eaa94b81` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:12.598077 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac6464-9416-48aa-b902-481b40253ab8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4ec9b09302d4c918ef638d794468ba3, user_id=8d3baa16824f40f3a4c783e210b262f4, role_id=b711a007d49d405fa3b1bf14eaa94b81)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:12.599500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac6464-9416-48aa-b902-481b40253ab8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:12.612710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 536/1079] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:08:12 2022] PUT /identity/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/8d3baa16824f40f3a4c783e210b262f4/roles/b711a007d49d405fa3b1bf14eaa94b81 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:08:12.838345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5e46c-87a9-4f5f-a688-c336fd60e245 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:12.838673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5e46c-87a9-4f5f-a688-c336fd60e245 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:12.838911 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5e46c-87a9-4f5f-a688-c336fd60e245 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:12.840319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 544/1080] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:12.845569 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f012789-9b91-4eb1-bdaf-6b15e61637ab None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:12.845863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f012789-9b91-4eb1-bdaf-6b15e61637ab None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:12.846144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f012789-9b91-4eb1-bdaf-6b15e61637ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:12.862804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6f012789-9b91-4eb1-bdaf-6b15e61637ab None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:12.863981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6f012789-9b91-4eb1-bdaf-6b15e61637ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:12.932160 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 537/1081] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:12.953042 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5e46c-87a9-4f5f-a688-c336fd60e245 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:12.977810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:13.057924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5e46c-87a9-4f5f-a688-c336fd60e245 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:13.092318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189bc34-8a16-4e79-8419-58bd171377ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:13.093301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189bc34-8a16-4e79-8419-58bd171377ef None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:13.093810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189bc34-8a16-4e79-8419-58bd171377ef None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:13.094301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189bc34-8a16-4e79-8419-58bd171377ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:13.166524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189bc34-8a16-4e79-8419-58bd171377ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:13.168342 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189bc34-8a16-4e79-8419-58bd171377ef None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:13.191923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:13.272102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 545/1082] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:08:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:08:13.274034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f012789-9b91-4eb1-bdaf-6b15e61637ab None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:13.318059 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:13.375941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f012789-9b91-4eb1-bdaf-6b15e61637ab None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:13.416198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e70df-bc1c-4b50-b295-deb14f4b7ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:13.416988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e70df-bc1c-4b50-b295-deb14f4b7ceb None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:13.417269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e70df-bc1c-4b50-b295-deb14f4b7ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:13.417542 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e70df-bc1c-4b50-b295-deb14f4b7ceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:13.427225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e70df-bc1c-4b50-b295-deb14f4b7ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:13.428707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e70df-bc1c-4b50-b295-deb14f4b7ceb None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-fb2e70df-bc1c-4b50-b295-deb14f4b7ceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:08:13.446523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:13.447407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:13.448347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:08:13.448347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:08:13.448347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:13.448347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:13.452122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 538/1083] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:13.453585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189bc34-8a16-4e79-8419-58bd171377ef None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:13.551991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189bc34-8a16-4e79-8419-58bd171377ef None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:13.575121 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:13.587996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508d90b-d20c-4dcf-a195-c64cb7d3051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:13.588701 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508d90b-d20c-4dcf-a195-c64cb7d3051f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:13.588986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508d90b-d20c-4dcf-a195-c64cb7d3051f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:13.589263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508d90b-d20c-4dcf-a195-c64cb7d3051f None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:13.589979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508d90b-d20c-4dcf-a195-c64cb7d3051f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:13.590287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508d90b-d20c-4dcf-a195-c64cb7d3051f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:13.591640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508d90b-d20c-4dcf-a195-c64cb7d3051f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:13.608096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 546/1084] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:14.173625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9fb13-ea84-40c3-944b-723728978f65 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:14.173918 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9fb13-ea84-40c3-944b-723728978f65 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:14.174202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9fb13-ea84-40c3-944b-723728978f65 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:14.175620 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 539/1085] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:14.180941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12236b-fe39-4416-a676-2edf78833dff None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:14.181381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12236b-fe39-4416-a676-2edf78833dff None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:14.181632 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12236b-fe39-4416-a676-2edf78833dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:14.197919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-7c12236b-fe39-4416-a676-2edf78833dff None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:14.199086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7c12236b-fe39-4416-a676-2edf78833dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:14.262821 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 547/1086] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:14.287297 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9fb13-ea84-40c3-944b-723728978f65 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:14.378597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:14.387053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9fb13-ea84-40c3-944b-723728978f65 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:14.421313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92e328-ef52-43f5-98b8-02a9ccdd1a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:14.422110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92e328-ef52-43f5-98b8-02a9ccdd1a7f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:14.422400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92e328-ef52-43f5-98b8-02a9ccdd1a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:14.422680 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92e328-ef52-43f5-98b8-02a9ccdd1a7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:14.433010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92e328-ef52-43f5-98b8-02a9ccdd1a7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-bd92e328-ef52-43f5-98b8-02a9ccdd1a7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:08:14.440660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:14.441810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:14.442658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 540/1087] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:08:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:14.448208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12236b-fe39-4416-a676-2edf78833dff None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:14.545625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12236b-fe39-4416-a676-2edf78833dff None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:14.559402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:14.579693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa842a-0568-4c98-bda0-8b71af325048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:14.580381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa842a-0568-4c98-bda0-8b71af325048 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:14.580670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa842a-0568-4c98-bda0-8b71af325048 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:14.580950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa842a-0568-4c98-bda0-8b71af325048 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:14.581686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa842a-0568-4c98-bda0-8b71af325048 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:14.581993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa842a-0568-4c98-bda0-8b71af325048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:14.583346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa842a-0568-4c98-bda0-8b71af325048 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:14.591615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 548/1088] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:08:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:14.597508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92e328-ef52-43f5-98b8-02a9ccdd1a7f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:14.698093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92e328-ef52-43f5-98b8-02a9ccdd1a7f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:14.731698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2dff6-e53f-4178-b38e-84ca5de9651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:14.732600 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2dff6-e53f-4178-b38e-84ca5de9651d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:14.732893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2dff6-e53f-4178-b38e-84ca5de9651d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:14.733171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2dff6-e53f-4178-b38e-84ca5de9651d None admin] PATH_INFO: `/v3/users/5b8d9ae7d08341e38d1b0477c6eb2eb5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:14.734364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2dff6-e53f-4178-b38e-84ca5de9651d None admin] RBAC: Authorizing `identity:get_user(user_id=5b8d9ae7d08341e38d1b0477c6eb2eb5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:14.735798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2dff6-e53f-4178-b38e-84ca5de9651d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:14.740093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 541/1089] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:14 2022] GET /identity/v3/users/5b8d9ae7d08341e38d1b0477c6eb2eb5 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:14.740215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a05de-96cc-4ae1-a006-8c0785c42fc0 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:14.740215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a05de-96cc-4ae1-a006-8c0785c42fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:14.740215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a05de-96cc-4ae1-a006-8c0785c42fc0 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:14.740215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 549/1090] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:08:14.743153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2dff6-e53f-4178-b38e-84ca5de9651d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:14.744938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c537a2-7f03-4be5-bf1a-5accd20be3c7 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:14.745231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c537a2-7f03-4be5-bf1a-5accd20be3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:14.745511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c537a2-7f03-4be5-bf1a-5accd20be3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:14.762080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-61c537a2-7f03-4be5-bf1a-5accd20be3c7 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:14.763260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-61c537a2-7f03-4be5-bf1a-5accd20be3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:14.769090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:14.829426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 550/1091] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:08:14.845055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2dff6-e53f-4178-b38e-84ca5de9651d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:14.852518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c537a2-7f03-4be5-bf1a-5accd20be3c7 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:14.878285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3472a3-4391-403a-9fd0-8b961ad7f66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:14.878936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3472a3-4391-403a-9fd0-8b961ad7f66b None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:14.879223 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3472a3-4391-403a-9fd0-8b961ad7f66b None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:14.879501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3472a3-4391-403a-9fd0-8b961ad7f66b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:14.944776 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3472a3-4391-403a-9fd0-8b961ad7f66b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:14.946261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3472a3-4391-403a-9fd0-8b961ad7f66b None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:14.948706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c537a2-7f03-4be5-bf1a-5accd20be3c7 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:14.950913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:14.969387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:14.982681 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb943b-1992-490f-99e7-980004d3eaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:14.983482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb943b-1992-490f-99e7-980004d3eaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:14.983765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb943b-1992-490f-99e7-980004d3eaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:14.984037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb943b-1992-490f-99e7-980004d3eaf3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:14.994707 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb943b-1992-490f-99e7-980004d3eaf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-64cb943b-1992-490f-99e7-980004d3eaf3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:08:15.002914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:15.003855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:15.004745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 551/1092] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jun 17 07:08:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:15.009936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb943b-1992-490f-99e7-980004d3eaf3 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:15.046339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 542/1093] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:08:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:08:15.052007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3472a3-4391-403a-9fd0-8b961ad7f66b None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:15.108799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb943b-1992-490f-99e7-980004d3eaf3 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:15.142193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45e26c-854d-479e-bfc5-e0ab3a7e3e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:15.142856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e26c-854d-479e-bfc5-e0ab3a7e3e9f None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:15.143152 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e26c-854d-479e-bfc5-e0ab3a7e3e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:15.143432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e26c-854d-479e-bfc5-e0ab3a7e3e9f None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:15.144204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45e26c-854d-479e-bfc5-e0ab3a7e3e9f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:15.144519 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45e26c-854d-479e-bfc5-e0ab3a7e3e9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:15.145886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45e26c-854d-479e-bfc5-e0ab3a7e3e9f None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:15.149464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3472a3-4391-403a-9fd0-8b961ad7f66b None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:15.153945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 552/1094] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jun 17 07:08:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:15.159416 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45e26c-854d-479e-bfc5-e0ab3a7e3e9f None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:15.174913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:15.187045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:15.187501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c94775-08d2-464c-95cd-72c9a99a2dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:15.188328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c94775-08d2-464c-95cd-72c9a99a2dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:15.188619 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c94775-08d2-464c-95cd-72c9a99a2dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:15.188885 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c94775-08d2-464c-95cd-72c9a99a2dd6 None admin] PATH_INFO: `/v3/projects/fa1e8038919344b08141ccb5518af99b` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:15.190108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c94775-08d2-464c-95cd-72c9a99a2dd6 None admin] RBAC: Authorizing `identity:get_project(project_id=fa1e8038919344b08141ccb5518af99b)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:15.191526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c94775-08d2-464c-95cd-72c9a99a2dd6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:15.193244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 543/1095] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:08:15 2022] GET /identity/v3/projects/fa1e8038919344b08141ccb5518af99b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:15.199122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c94775-08d2-464c-95cd-72c9a99a2dd6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:15.259248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45e26c-854d-479e-bfc5-e0ab3a7e3e9f None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:15.292961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dd4f3-f2b1-4bcf-8e4c-0f7630ca5bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:15.293735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dd4f3-f2b1-4bcf-8e4c-0f7630ca5bbf None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:15.294016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dd4f3-f2b1-4bcf-8e4c-0f7630ca5bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:15.294287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dd4f3-f2b1-4bcf-8e4c-0f7630ca5bbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:15.294606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c94775-08d2-464c-95cd-72c9a99a2dd6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:15.304505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4dd4f3-f2b1-4bcf-8e4c-0f7630ca5bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:15.306451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4dd4f3-f2b1-4bcf-8e4c-0f7630ca5bbf None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-5d4dd4f3-f2b1-4bcf-8e4c-0f7630ca5bbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:08:15.323251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:15.324351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:15.325648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:08:15.325648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:08:15.325648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:15.325648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:15.325648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 553/1096] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:15.329918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dd4f3-f2b1-4bcf-8e4c-0f7630ca5bbf None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:15.330745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c230f-f231-49b4-b82b-e1ba8093075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:15.331394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c230f-f231-49b4-b82b-e1ba8093075e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:15.331682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c230f-f231-49b4-b82b-e1ba8093075e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:15.331965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c230f-f231-49b4-b82b-e1ba8093075e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:15.332831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c230f-f231-49b4-b82b-e1ba8093075e None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=fa1e8038919344b08141ccb5518af99b, user.id=5b8d9ae7d08341e38d1b0477c6eb2eb5) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:15.333151 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c230f-f231-49b4-b82b-e1ba8093075e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:15.334598 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c230f-f231-49b4-b82b-e1ba8093075e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:15.341162 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 544/1097] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:08:15 2022] GET /identity/v3/role_assignments?user.id=5b8d9ae7d08341e38d1b0477c6eb2eb5&scope.project.id=fa1e8038919344b08141ccb5518af99b&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:15.416892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:15.438783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dd4f3-f2b1-4bcf-8e4c-0f7630ca5bbf None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:15.473631 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51459c-3be7-4656-b4e6-592e8ebf6790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:15.474225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51459c-3be7-4656-b4e6-592e8ebf6790 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:15.474514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51459c-3be7-4656-b4e6-592e8ebf6790 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:15.474796 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51459c-3be7-4656-b4e6-592e8ebf6790 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:15.475529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51459c-3be7-4656-b4e6-592e8ebf6790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:15.475842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51459c-3be7-4656-b4e6-592e8ebf6790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:15.477832 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51459c-3be7-4656-b4e6-592e8ebf6790 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:15.490498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 554/1098] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:15.496259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c230f-f231-49b4-b82b-e1ba8093075e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:15.557181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:15.600962 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c230f-f231-49b4-b82b-e1ba8093075e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:15.633782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366131e-a1df-43f5-a163-2e6f68327793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:15.634573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366131e-a1df-43f5-a163-2e6f68327793 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:15.634861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366131e-a1df-43f5-a163-2e6f68327793 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:15.635144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366131e-a1df-43f5-a163-2e6f68327793 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:15.642448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366131e-a1df-43f5-a163-2e6f68327793 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:15.643892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366131e-a1df-43f5-a163-2e6f68327793 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-6366131e-a1df-43f5-a163-2e6f68327793 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:08:15.658393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:15.659516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:15.660846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:15.660846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:15.660846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:08:15.660846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:08:15.660846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:08:15.660846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:08:15.660846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:08:15.660846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:08:15.660846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 07:08:15.660846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:15.660846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 545/1099] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jun 17 07:08:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:15.665706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51459c-3be7-4656-b4e6-592e8ebf6790 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:15.760926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51459c-3be7-4656-b4e6-592e8ebf6790 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:15.793554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1caaa-a1e9-4205-b340-20d988ad3901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:15.794214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1caaa-a1e9-4205-b340-20d988ad3901 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:15.794496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1caaa-a1e9-4205-b340-20d988ad3901 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:15.794770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1caaa-a1e9-4205-b340-20d988ad3901 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:15.795594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1caaa-a1e9-4205-b340-20d988ad3901 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:15.795899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1caaa-a1e9-4205-b340-20d988ad3901 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:15.797350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1caaa-a1e9-4205-b340-20d988ad3901 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:15.805490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:15.815639 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 555/1100] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jun 17 07:08:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:08:15.821272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366131e-a1df-43f5-a163-2e6f68327793 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:15.916999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366131e-a1df-43f5-a163-2e6f68327793 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:15.930385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:15.950477 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee57bee-35f6-46b9-8c24-dcce1d06fe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:15.951118 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee57bee-35f6-46b9-8c24-dcce1d06fe6e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:15.951402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee57bee-35f6-46b9-8c24-dcce1d06fe6e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:15.951677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee57bee-35f6-46b9-8c24-dcce1d06fe6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:16.015297 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee57bee-35f6-46b9-8c24-dcce1d06fe6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:16.016698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee57bee-35f6-46b9-8c24-dcce1d06fe6e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:16.127222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 546/1101] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:08:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 07:08:16.132923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1caaa-a1e9-4205-b340-20d988ad3901 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:16.229814 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1caaa-a1e9-4205-b340-20d988ad3901 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:16.261686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530537cc-16ff-4501-a846-0e3a21aa5791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:16.262465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530537cc-16ff-4501-a846-0e3a21aa5791 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:16.262749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530537cc-16ff-4501-a846-0e3a21aa5791 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:16.263104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530537cc-16ff-4501-a846-0e3a21aa5791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:16.272854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530537cc-16ff-4501-a846-0e3a21aa5791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:16.274966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530537cc-16ff-4501-a846-0e3a21aa5791 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-530537cc-16ff-4501-a846-0e3a21aa5791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:08:16.292381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:16.293244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:16.294123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:08:16.294123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:08:16.294123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:08:16.294123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:16.294123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 556/1102] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:16.300167 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee57bee-35f6-46b9-8c24-dcce1d06fe6e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:16.314418 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:16.400428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee57bee-35f6-46b9-8c24-dcce1d06fe6e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:16.433576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df75460-0d7a-45bc-8cba-ea6eb5d579eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:16.434246 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df75460-0d7a-45bc-8cba-ea6eb5d579eb None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:16.434536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df75460-0d7a-45bc-8cba-ea6eb5d579eb None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:16.434814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df75460-0d7a-45bc-8cba-ea6eb5d579eb None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:16.435548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df75460-0d7a-45bc-8cba-ea6eb5d579eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:16.435857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df75460-0d7a-45bc-8cba-ea6eb5d579eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:16.437308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df75460-0d7a-45bc-8cba-ea6eb5d579eb None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:16.449726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 547/1103] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:16.455310 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530537cc-16ff-4501-a846-0e3a21aa5791 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:16.503064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:16.552135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530537cc-16ff-4501-a846-0e3a21aa5791 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:16.585673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6aa5a-c255-4c26-b648-78d26299cb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:16.587017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6aa5a-c255-4c26-b648-78d26299cb75 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:16.587302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6aa5a-c255-4c26-b648-78d26299cb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:16.587576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6aa5a-c255-4c26-b648-78d26299cb75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:16.593978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6aa5a-c255-4c26-b648-78d26299cb75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:16.595404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6aa5a-c255-4c26-b648-78d26299cb75 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-76a6aa5a-c255-4c26-b648-78d26299cb75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:08:16.609215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:16.610101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:16.611148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:08:16.611148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:08:16.611148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:08:16.611148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:08:16.611148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:08:16.611148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:08:16.611148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:08:16.611148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:16.611148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 557/1104] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:08:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:08:16.616121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df75460-0d7a-45bc-8cba-ea6eb5d579eb None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:16.691222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:16.715187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df75460-0d7a-45bc-8cba-ea6eb5d579eb None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:16.747457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e629ad-d876-4e1d-ae68-977610919658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:16.748127 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e629ad-d876-4e1d-ae68-977610919658 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:16.748412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e629ad-d876-4e1d-ae68-977610919658 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:16.748686 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e629ad-d876-4e1d-ae68-977610919658 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:16.749456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e629ad-d876-4e1d-ae68-977610919658 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:16.749765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e629ad-d876-4e1d-ae68-977610919658 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:16.751211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e629ad-d876-4e1d-ae68-977610919658 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:16.762843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 548/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:08:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:16.768612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6aa5a-c255-4c26-b648-78d26299cb75 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:16.863859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6aa5a-c255-4c26-b648-78d26299cb75 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:16.872453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:16.896120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973c445-5aa1-4242-b40c-84c856a057de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:16.896763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973c445-5aa1-4242-b40c-84c856a057de None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:16.897046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973c445-5aa1-4242-b40c-84c856a057de None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:16.897321 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973c445-5aa1-4242-b40c-84c856a057de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:16.898139 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973c445-5aa1-4242-b40c-84c856a057de None admin] RBAC: Adding query filter params (role.id=b711a007d49d405fa3b1bf14eaa94b81, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=8d3baa16824f40f3a4c783e210b262f4) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:16.898965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973c445-5aa1-4242-b40c-84c856a057de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:16.900655 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973c445-5aa1-4242-b40c-84c856a057de None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:16.906290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 558/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:08:16 2022] GET /identity/v3/role_assignments?user.id=8d3baa16824f40f3a4c783e210b262f4&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=b711a007d49d405fa3b1bf14eaa94b81 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:17.481041 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f658b4e-8f16-480f-8638-788c3a7c1794 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:17.481580 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f658b4e-8f16-480f-8638-788c3a7c1794 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:17.481671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f658b4e-8f16-480f-8638-788c3a7c1794 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:17.483042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 549/1107] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:17.488334 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881432b0-b829-410c-ba2b-6b374af7679a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:17.488760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881432b0-b829-410c-ba2b-6b374af7679a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:17.488901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881432b0-b829-410c-ba2b-6b374af7679a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:17.506002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-881432b0-b829-410c-ba2b-6b374af7679a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:17.507187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-881432b0-b829-410c-ba2b-6b374af7679a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:17.574388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 559/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:17.598901 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f658b4e-8f16-480f-8638-788c3a7c1794 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:17.701759 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f658b4e-8f16-480f-8638-788c3a7c1794 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:17.735725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a21287-d39f-4c6f-9129-ca0722c74ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:17.736546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a21287-d39f-4c6f-9129-ca0722c74ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:17.736832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a21287-d39f-4c6f-9129-ca0722c74ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:17.737107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a21287-d39f-4c6f-9129-ca0722c74ad5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:17.738273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a21287-d39f-4c6f-9129-ca0722c74ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:17.739702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a21287-d39f-4c6f-9129-ca0722c74ad5 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:17.741402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 550/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:17.747649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881432b0-b829-410c-ba2b-6b374af7679a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:17.842802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881432b0-b829-410c-ba2b-6b374af7679a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:17.844271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:17.876175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cff13-4650-4dd8-8008-8e21bf879769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:17.876820 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cff13-4650-4dd8-8008-8e21bf879769 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:17.877103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cff13-4650-4dd8-8008-8e21bf879769 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:17.877379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cff13-4650-4dd8-8008-8e21bf879769 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:17.878070 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cff13-4650-4dd8-8008-8e21bf879769 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:17.879472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cff13-4650-4dd8-8008-8e21bf879769 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:17.899378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 560/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:08:17 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:08:20.005661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a689e-182c-44a0-a19e-5b035549637f None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:20.006258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a689e-182c-44a0-a19e-5b035549637f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:20.006258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a689e-182c-44a0-a19e-5b035549637f None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:20.007627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 551/1111] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:20.013243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16685bbd-d7a8-460a-ad13-e15a14884d0a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:20.013632 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16685bbd-d7a8-460a-ad13-e15a14884d0a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:20.013800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16685bbd-d7a8-460a-ad13-e15a14884d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:20.030166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-16685bbd-d7a8-460a-ad13-e15a14884d0a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:20.031329 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-16685bbd-d7a8-460a-ad13-e15a14884d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:20.094064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 561/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:20.116239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a689e-182c-44a0-a19e-5b035549637f None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:20.119644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:20.219956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a689e-182c-44a0-a19e-5b035549637f None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:20.253504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832a6d2-54fd-4b12-8770-914f0d8bcc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:20.254351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832a6d2-54fd-4b12-8770-914f0d8bcc61 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:20.254637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832a6d2-54fd-4b12-8770-914f0d8bcc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:20.254913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832a6d2-54fd-4b12-8770-914f0d8bcc61 None admin] PATH_INFO: `/v3/roles/22d7b181c7e041329c59c3eb6b492cd7` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:20.256457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832a6d2-54fd-4b12-8770-914f0d8bcc61 None admin] RBAC: Authorizing `identity:get_role(role_id=22d7b181c7e041329c59c3eb6b492cd7)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:20.257805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832a6d2-54fd-4b12-8770-914f0d8bcc61 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:20.259149 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 552/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:20 2022] GET /identity/v3/roles/22d7b181c7e041329c59c3eb6b492cd7 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:20.264897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16685bbd-d7a8-460a-ad13-e15a14884d0a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:20.349862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:20.363736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16685bbd-d7a8-460a-ad13-e15a14884d0a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:20.396511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22ec0f-8b39-4f31-838e-7299dc77d18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:20.397377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22ec0f-8b39-4f31-838e-7299dc77d18a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:20.397660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22ec0f-8b39-4f31-838e-7299dc77d18a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:20.397936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22ec0f-8b39-4f31-838e-7299dc77d18a None admin] PATH_INFO: `/v3/users/ac312b5b34374e9bb6cca16f71cac6db` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:20.411586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22ec0f-8b39-4f31-838e-7299dc77d18a None admin] RBAC: Authorizing `identity:get_user(user_id=ac312b5b34374e9bb6cca16f71cac6db)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:20.413029 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22ec0f-8b39-4f31-838e-7299dc77d18a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:20.414659 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 562/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:20 2022] GET /identity/v3/users/ac312b5b34374e9bb6cca16f71cac6db => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:20.420339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832a6d2-54fd-4b12-8770-914f0d8bcc61 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:20.486560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:20.515680 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832a6d2-54fd-4b12-8770-914f0d8bcc61 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:20.548190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71442cb8-d9e5-4414-8752-41894cd6d58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:20.548821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442cb8-d9e5-4414-8752-41894cd6d58f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:20.549103 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442cb8-d9e5-4414-8752-41894cd6d58f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:20.549375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442cb8-d9e5-4414-8752-41894cd6d58f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:20.612032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71442cb8-d9e5-4414-8752-41894cd6d58f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:20.613481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71442cb8-d9e5-4414-8752-41894cd6d58f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:20.677717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:20.708360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 553/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:08:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:08:20.714629 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22ec0f-8b39-4f31-838e-7299dc77d18a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:20.811318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22ec0f-8b39-4f31-838e-7299dc77d18a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:20.843824 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c669f-f715-4e60-84d8-c42913dd11ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:20.844694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c669f-f715-4e60-84d8-c42913dd11ef None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:20.844980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c669f-f715-4e60-84d8-c42913dd11ef None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:20.845258 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c669f-f715-4e60-84d8-c42913dd11ef None admin] PATH_INFO: `/v3/projects/fa1e8038919344b08141ccb5518af99b` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:20.846420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c669f-f715-4e60-84d8-c42913dd11ef None admin] RBAC: Authorizing `identity:get_project(project_id=fa1e8038919344b08141ccb5518af99b)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:20.847844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c669f-f715-4e60-84d8-c42913dd11ef None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:20.849544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 563/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:08:20 2022] GET /identity/v3/projects/fa1e8038919344b08141ccb5518af99b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:20.855998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71442cb8-d9e5-4414-8752-41894cd6d58f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:20.949871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71442cb8-d9e5-4414-8752-41894cd6d58f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:20.982076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848dfd5-e7b2-4efd-81ce-3e716f641548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:20.982710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848dfd5-e7b2-4efd-81ce-3e716f641548 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:20.982996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848dfd5-e7b2-4efd-81ce-3e716f641548 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:20.983273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848dfd5-e7b2-4efd-81ce-3e716f641548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:20.984113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848dfd5-e7b2-4efd-81ce-3e716f641548 None admin] RBAC: Adding query filter params (role.id=22d7b181c7e041329c59c3eb6b492cd7, scope.project.id=fa1e8038919344b08141ccb5518af99b, user.id=ac312b5b34374e9bb6cca16f71cac6db) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:20.984433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848dfd5-e7b2-4efd-81ce-3e716f641548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:20.985855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848dfd5-e7b2-4efd-81ce-3e716f641548 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:20.991085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 554/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:08:20 2022] GET /identity/v3/role_assignments?user.id=ac312b5b34374e9bb6cca16f71cac6db&scope.project.id=fa1e8038919344b08141ccb5518af99b&role.id=22d7b181c7e041329c59c3eb6b492cd7 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:23.048743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53f100-6665-42e3-b56c-7eaaf2037279 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:23.049290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53f100-6665-42e3-b56c-7eaaf2037279 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:23.049290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53f100-6665-42e3-b56c-7eaaf2037279 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:23.050680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 564/1118] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:23.055877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85579ccc-7ef7-4ea0-8721-3a6b15b0e0ca None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:23.056332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85579ccc-7ef7-4ea0-8721-3a6b15b0e0ca None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:23.056479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85579ccc-7ef7-4ea0-8721-3a6b15b0e0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:23.072778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-85579ccc-7ef7-4ea0-8721-3a6b15b0e0ca None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:23.073966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-85579ccc-7ef7-4ea0-8721-3a6b15b0e0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:23.099543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:23.138817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 555/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:23.161678 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53f100-6665-42e3-b56c-7eaaf2037279 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:23.177069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:23.260932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53f100-6665-42e3-b56c-7eaaf2037279 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:23.294102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85602d-f057-44ea-980d-60a59adedb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:23.294940 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85602d-f057-44ea-980d-60a59adedb40 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:23.295227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85602d-f057-44ea-980d-60a59adedb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:23.295503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85602d-f057-44ea-980d-60a59adedb40 None admin] PATH_INFO: `/v3/roles/22d7b181c7e041329c59c3eb6b492cd7` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:23.296982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85602d-f057-44ea-980d-60a59adedb40 None admin] RBAC: Authorizing `identity:get_role(role_id=22d7b181c7e041329c59c3eb6b492cd7)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:23.298350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85602d-f057-44ea-980d-60a59adedb40 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:23.299744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 565/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:23 2022] GET /identity/v3/roles/22d7b181c7e041329c59c3eb6b492cd7 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:23.305995 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85579ccc-7ef7-4ea0-8721-3a6b15b0e0ca None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:23.402113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85579ccc-7ef7-4ea0-8721-3a6b15b0e0ca None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:23.434056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be8f57-5b3b-4eb0-9f6e-ab443dc16a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:23.434914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be8f57-5b3b-4eb0-9f6e-ab443dc16a62 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:23.435199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be8f57-5b3b-4eb0-9f6e-ab443dc16a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:23.435476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be8f57-5b3b-4eb0-9f6e-ab443dc16a62 None admin] PATH_INFO: `/v3/users/ac312b5b34374e9bb6cca16f71cac6db` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:23.436669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be8f57-5b3b-4eb0-9f6e-ab443dc16a62 None admin] RBAC: Authorizing `identity:get_user(user_id=ac312b5b34374e9bb6cca16f71cac6db)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:23.438105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be8f57-5b3b-4eb0-9f6e-ab443dc16a62 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:23.439707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 556/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:23 2022] GET /identity/v3/users/ac312b5b34374e9bb6cca16f71cac6db => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:23.445962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85602d-f057-44ea-980d-60a59adedb40 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:23.525617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:23.543460 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85602d-f057-44ea-980d-60a59adedb40 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:23.575632 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f65d6b-1fb7-4dbb-9f2b-17368a8e573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:23.576433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f65d6b-1fb7-4dbb-9f2b-17368a8e573c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:23.576717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f65d6b-1fb7-4dbb-9f2b-17368a8e573c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:23.576989 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f65d6b-1fb7-4dbb-9f2b-17368a8e573c None admin] PATH_INFO: `/v3/projects/fa1e8038919344b08141ccb5518af99b` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:23.578147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f65d6b-1fb7-4dbb-9f2b-17368a8e573c None admin] RBAC: Authorizing `identity:get_project(project_id=fa1e8038919344b08141ccb5518af99b)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:23.579560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f65d6b-1fb7-4dbb-9f2b-17368a8e573c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:23.581239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 566/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:08:23 2022] GET /identity/v3/projects/fa1e8038919344b08141ccb5518af99b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:23.587539 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be8f57-5b3b-4eb0-9f6e-ab443dc16a62 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:23.618038 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:23.684471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be8f57-5b3b-4eb0-9f6e-ab443dc16a62 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:23.716774 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2603c-f09e-4a91-b96e-e5b6031e0b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:23.717641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2603c-f09e-4a91-b96e-e5b6031e0b46 None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:23.717927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2603c-f09e-4a91-b96e-e5b6031e0b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:23.718205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2603c-f09e-4a91-b96e-e5b6031e0b46 None admin] PATH_INFO: `/v3/projects/fa1e8038919344b08141ccb5518af99b/users/ac312b5b34374e9bb6cca16f71cac6db/roles/22d7b181c7e041329c59c3eb6b492cd7` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:23.720406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2603c-f09e-4a91-b96e-e5b6031e0b46 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa1e8038919344b08141ccb5518af99b, user_id=ac312b5b34374e9bb6cca16f71cac6db, role_id=22d7b181c7e041329c59c3eb6b492cd7)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:23.721832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2603c-f09e-4a91-b96e-e5b6031e0b46 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:23.734666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 557/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:08:23 2022] PUT /identity/v3/projects/fa1e8038919344b08141ccb5518af99b/users/ac312b5b34374e9bb6cca16f71cac6db/roles/22d7b181c7e041329c59c3eb6b492cd7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:08:25.775649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47dc72-8b7e-4a03-b244-34f2f8d2a1f4 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:25.776201 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47dc72-8b7e-4a03-b244-34f2f8d2a1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:25.776315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47dc72-8b7e-4a03-b244-34f2f8d2a1f4 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:25.777607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 567/1124] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:25.782777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b9be-92d4-4681-a23b-3bc20162e56b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:25.783144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b9be-92d4-4681-a23b-3bc20162e56b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:25.783339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b9be-92d4-4681-a23b-3bc20162e56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:25.800025 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-a4e1b9be-92d4-4681-a23b-3bc20162e56b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:25.801228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-a4e1b9be-92d4-4681-a23b-3bc20162e56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:25.863779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 558/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:25.886196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47dc72-8b7e-4a03-b244-34f2f8d2a1f4 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:25.982312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47dc72-8b7e-4a03-b244-34f2f8d2a1f4 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:26.014248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49986a-1f02-48fe-8dfd-6adff13389c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:26.015071 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49986a-1f02-48fe-8dfd-6adff13389c7 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:26.015353 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49986a-1f02-48fe-8dfd-6adff13389c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:26.015627 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49986a-1f02-48fe-8dfd-6adff13389c7 None admin] PATH_INFO: `/v3/roles/22d7b181c7e041329c59c3eb6b492cd7` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:26.017082 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49986a-1f02-48fe-8dfd-6adff13389c7 None admin] RBAC: Authorizing `identity:get_role(role_id=22d7b181c7e041329c59c3eb6b492cd7)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:26.018436 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49986a-1f02-48fe-8dfd-6adff13389c7 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:26.019759 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 568/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:25 2022] GET /identity/v3/roles/22d7b181c7e041329c59c3eb6b492cd7 => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:26.026025 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1b9be-92d4-4681-a23b-3bc20162e56b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:26.048887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:26.121442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1b9be-92d4-4681-a23b-3bc20162e56b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:26.154982 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f53b1e-63fc-4f5c-9f74-c464ff31696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:26.155843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f53b1e-63fc-4f5c-9f74-c464ff31696a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:26.156157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f53b1e-63fc-4f5c-9f74-c464ff31696a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:26.156439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f53b1e-63fc-4f5c-9f74-c464ff31696a None admin] PATH_INFO: `/v3/users/ac312b5b34374e9bb6cca16f71cac6db` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:26.157618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f53b1e-63fc-4f5c-9f74-c464ff31696a None admin] RBAC: Authorizing `identity:get_user(user_id=ac312b5b34374e9bb6cca16f71cac6db)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:26.159052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f53b1e-63fc-4f5c-9f74-c464ff31696a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:26.160821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 559/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:26 2022] GET /identity/v3/users/ac312b5b34374e9bb6cca16f71cac6db => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:26.166303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49986a-1f02-48fe-8dfd-6adff13389c7 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:26.184493 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:26.261896 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49986a-1f02-48fe-8dfd-6adff13389c7 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:26.294595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d767803-a892-4a72-85c6-4d8d712be5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:26.295163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d767803-a892-4a72-85c6-4d8d712be5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:26.295447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d767803-a892-4a72-85c6-4d8d712be5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:26.295720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d767803-a892-4a72-85c6-4d8d712be5c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:26.358699 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d767803-a892-4a72-85c6-4d8d712be5c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:26.360147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d767803-a892-4a72-85c6-4d8d712be5c1 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:26.377960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:26.457002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 569/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:08:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:08:26.463713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f53b1e-63fc-4f5c-9f74-c464ff31696a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:26.549749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:26.559941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f53b1e-63fc-4f5c-9f74-c464ff31696a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:26.592390 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6acd36-1647-4f09-9bf9-fee5a3934cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:26.593229 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6acd36-1647-4f09-9bf9-fee5a3934cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:26.593515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6acd36-1647-4f09-9bf9-fee5a3934cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:26.593792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6acd36-1647-4f09-9bf9-fee5a3934cf7 None admin] PATH_INFO: `/v3/projects/fa1e8038919344b08141ccb5518af99b` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:26.594952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6acd36-1647-4f09-9bf9-fee5a3934cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=fa1e8038919344b08141ccb5518af99b)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:26.596404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6acd36-1647-4f09-9bf9-fee5a3934cf7 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:26.598057 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 560/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:08:26 2022] GET /identity/v3/projects/fa1e8038919344b08141ccb5518af99b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:26.604014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d767803-a892-4a72-85c6-4d8d712be5c1 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:26.699098 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d767803-a892-4a72-85c6-4d8d712be5c1 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:26.715690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:26.732198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5d1c0-7aeb-47ff-8704-6b58f89f5152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:26.732833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5d1c0-7aeb-47ff-8704-6b58f89f5152 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:26.733116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5d1c0-7aeb-47ff-8704-6b58f89f5152 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:26.733391 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5d1c0-7aeb-47ff-8704-6b58f89f5152 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:26.734206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5d1c0-7aeb-47ff-8704-6b58f89f5152 None admin] RBAC: Adding query filter params (role.id=22d7b181c7e041329c59c3eb6b492cd7, scope.project.id=fa1e8038919344b08141ccb5518af99b, user.id=ac312b5b34374e9bb6cca16f71cac6db) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:26.734515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5d1c0-7aeb-47ff-8704-6b58f89f5152 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:26.735919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5d1c0-7aeb-47ff-8704-6b58f89f5152 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:26.741509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 570/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:08:26 2022] GET /identity/v3/role_assignments?user.id=ac312b5b34374e9bb6cca16f71cac6db&scope.project.id=fa1e8038919344b08141ccb5518af99b&role.id=22d7b181c7e041329c59c3eb6b492cd7 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:28.849284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efab174-9c5b-4ef8-9811-26105017a0c9 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:28.849814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efab174-9c5b-4ef8-9811-26105017a0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:28.849814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efab174-9c5b-4ef8-9811-26105017a0c9 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:28.851251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 561/1131] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:28.856472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d6b00-2122-438b-91bd-c25ab572c900 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:28.856853 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d6b00-2122-438b-91bd-c25ab572c900 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:28.857122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d6b00-2122-438b-91bd-c25ab572c900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:28.873925 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-030d6b00-2122-438b-91bd-c25ab572c900 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:28.875100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-030d6b00-2122-438b-91bd-c25ab572c900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:28.938964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 571/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:28.962340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efab174-9c5b-4ef8-9811-26105017a0c9 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:29.058167 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efab174-9c5b-4ef8-9811-26105017a0c9 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:29.090109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b29b4-be74-4594-9a09-cf17fcbd3f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:29.090876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b29b4-be74-4594-9a09-cf17fcbd3f39 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:29.091159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b29b4-be74-4594-9a09-cf17fcbd3f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:29.091434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b29b4-be74-4594-9a09-cf17fcbd3f39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:29.092710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b29b4-be74-4594-9a09-cf17fcbd3f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:29.094130 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b29b4-be74-4594-9a09-cf17fcbd3f39 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:29.095687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 562/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:29.101319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d6b00-2122-438b-91bd-c25ab572c900 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:29.197161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d6b00-2122-438b-91bd-c25ab572c900 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:29.209652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:29.230596 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f90d7e-ddbe-4f08-8dbc-d0605c04a989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:29.231245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f90d7e-ddbe-4f08-8dbc-d0605c04a989 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:29.231533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f90d7e-ddbe-4f08-8dbc-d0605c04a989 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:29.231815 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f90d7e-ddbe-4f08-8dbc-d0605c04a989 None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:29.232551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f90d7e-ddbe-4f08-8dbc-d0605c04a989 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:29.233977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f90d7e-ddbe-4f08-8dbc-d0605c04a989 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:29.258070 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 572/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:08:29 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:08:31.363359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c79986-be4b-435b-94d8-b0af7c029a2a None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:31.363892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c79986-be4b-435b-94d8-b0af7c029a2a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:31.363892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c79986-be4b-435b-94d8-b0af7c029a2a None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:31.365308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 563/1135] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:31.371411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560813c-821e-4672-938f-7d154281ca92 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:31.371802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560813c-821e-4672-938f-7d154281ca92 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:31.371891 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560813c-821e-4672-938f-7d154281ca92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:31.388853 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c560813c-821e-4672-938f-7d154281ca92 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:31.390038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c560813c-821e-4672-938f-7d154281ca92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:31.455098 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 573/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:08:31.479496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c79986-be4b-435b-94d8-b0af7c029a2a None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:31.505949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:31.578857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c79986-be4b-435b-94d8-b0af7c029a2a None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:31.611317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac151b8b-9e6b-436f-9f69-3daf1eb7e4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:31.612114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac151b8b-9e6b-436f-9f69-3daf1eb7e4da None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:31.612406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac151b8b-9e6b-436f-9f69-3daf1eb7e4da None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:31.612685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac151b8b-9e6b-436f-9f69-3daf1eb7e4da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:31.613912 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac151b8b-9e6b-436f-9f69-3daf1eb7e4da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:31.615298 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac151b8b-9e6b-436f-9f69-3daf1eb7e4da None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:31.616931 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 564/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:08:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:31.622477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560813c-821e-4672-938f-7d154281ca92 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:31.711441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:31.719147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560813c-821e-4672-938f-7d154281ca92 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:31.751772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671dd89-a3bc-4687-82c0-3cb0769b4bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:31.752457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671dd89-a3bc-4687-82c0-3cb0769b4bb2 None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:31.752748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671dd89-a3bc-4687-82c0-3cb0769b4bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:31.753029 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671dd89-a3bc-4687-82c0-3cb0769b4bb2 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:31.753742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671dd89-a3bc-4687-82c0-3cb0769b4bb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:31.755172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671dd89-a3bc-4687-82c0-3cb0769b4bb2 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:31.774014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 574/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:08:31 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:08:33.873575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea93db-e65e-4289-b981-fca9f36ac96f None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:33.874134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea93db-e65e-4289-b981-fca9f36ac96f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:33.874134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea93db-e65e-4289-b981-fca9f36ac96f None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:33.875486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 565/1139] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:33.880540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff582b41-95ba-4ad1-a834-43582f827eec None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:33.880935 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff582b41-95ba-4ad1-a834-43582f827eec None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:33.881111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff582b41-95ba-4ad1-a834-43582f827eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:33.898569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ff582b41-95ba-4ad1-a834-43582f827eec None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:33.900004 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ff582b41-95ba-4ad1-a834-43582f827eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:33.964683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 575/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:33.987256 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea93db-e65e-4289-b981-fca9f36ac96f None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:34.079433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:34.087686 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea93db-e65e-4289-b981-fca9f36ac96f None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:34.120163 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe37910-c5b2-41aa-8a5d-b2712ae70336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:34.120942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe37910-c5b2-41aa-8a5d-b2712ae70336 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:34.121228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe37910-c5b2-41aa-8a5d-b2712ae70336 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:34.121509 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe37910-c5b2-41aa-8a5d-b2712ae70336 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:34.131327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe37910-c5b2-41aa-8a5d-b2712ae70336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-9fe37910-c5b2-41aa-8a5d-b2712ae70336 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:08:34.139498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:34.140719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:34.141678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 566/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:08:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:34.146462 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff582b41-95ba-4ad1-a834-43582f827eec None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:34.223949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:34.244383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff582b41-95ba-4ad1-a834-43582f827eec None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:34.277723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99aad9f-4e08-42dc-bc80-95c999588c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:34.278374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99aad9f-4e08-42dc-bc80-95c999588c6e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:34.278660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99aad9f-4e08-42dc-bc80-95c999588c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:34.278938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99aad9f-4e08-42dc-bc80-95c999588c6e None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:34.279672 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99aad9f-4e08-42dc-bc80-95c999588c6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:34.279983 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99aad9f-4e08-42dc-bc80-95c999588c6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:34.281564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99aad9f-4e08-42dc-bc80-95c999588c6e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:34.290245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 576/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:08:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:34.296006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe37910-c5b2-41aa-8a5d-b2712ae70336 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:34.392181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe37910-c5b2-41aa-8a5d-b2712ae70336 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:34.424956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4184e-67ef-463b-8654-cf9b06551353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:34.425819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4184e-67ef-463b-8654-cf9b06551353 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:34.426106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4184e-67ef-463b-8654-cf9b06551353 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:34.426388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4184e-67ef-463b-8654-cf9b06551353 None admin] PATH_INFO: `/v3/users/5bbb0aba6a394e67998daa1cef443659` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:34.441012 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4184e-67ef-463b-8654-cf9b06551353 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbb0aba6a394e67998daa1cef443659)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:34.442454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4184e-67ef-463b-8654-cf9b06551353 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:34.444129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 567/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:34 2022] GET /identity/v3/users/5bbb0aba6a394e67998daa1cef443659 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:34.450265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99aad9f-4e08-42dc-bc80-95c999588c6e None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:34.545981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99aad9f-4e08-42dc-bc80-95c999588c6e None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:34.578013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfc6f5-1eb8-4d23-ab18-13c1bfeca540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:34.578642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfc6f5-1eb8-4d23-ab18-13c1bfeca540 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:34.578923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfc6f5-1eb8-4d23-ab18-13c1bfeca540 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:34.579197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfc6f5-1eb8-4d23-ab18-13c1bfeca540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:34.590900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:34.643662 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfc6f5-1eb8-4d23-ab18-13c1bfeca540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:34.645055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfc6f5-1eb8-4d23-ab18-13c1bfeca540 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:34.737445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 577/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:08:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:08:34.743805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4184e-67ef-463b-8654-cf9b06551353 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:34.758564 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:34.839671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4184e-67ef-463b-8654-cf9b06551353 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:34.871967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e40ec-3f68-40ff-bf57-13cbc7623412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:34.872829 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e40ec-3f68-40ff-bf57-13cbc7623412 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:34.873115 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e40ec-3f68-40ff-bf57-13cbc7623412 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:34.873392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e40ec-3f68-40ff-bf57-13cbc7623412 None admin] PATH_INFO: `/v3/projects/4d9591df09ea465c80603f23837b92a3` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:34.874566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e40ec-3f68-40ff-bf57-13cbc7623412 None admin] RBAC: Authorizing `identity:get_project(project_id=4d9591df09ea465c80603f23837b92a3)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:34.875992 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e40ec-3f68-40ff-bf57-13cbc7623412 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:34.877688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 568/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:08:34 2022] GET /identity/v3/projects/4d9591df09ea465c80603f23837b92a3 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:34.883587 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfc6f5-1eb8-4d23-ab18-13c1bfeca540 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:34.924979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:34.977889 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfc6f5-1eb8-4d23-ab18-13c1bfeca540 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:35.009879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b50d33-c348-4311-91ac-b173c8c52a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:35.010507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b50d33-c348-4311-91ac-b173c8c52a04 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:35.010789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b50d33-c348-4311-91ac-b173c8c52a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:35.011065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b50d33-c348-4311-91ac-b173c8c52a04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:35.011880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b50d33-c348-4311-91ac-b173c8c52a04 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=4d9591df09ea465c80603f23837b92a3, user.id=5bbb0aba6a394e67998daa1cef443659) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:35.012221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b50d33-c348-4311-91ac-b173c8c52a04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:35.013626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b50d33-c348-4311-91ac-b173c8c52a04 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:35.018801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 578/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:08:34 2022] GET /identity/v3/role_assignments?user.id=5bbb0aba6a394e67998daa1cef443659&scope.project.id=4d9591df09ea465c80603f23837b92a3&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:37.077250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea39a6-990c-4d47-86d9-fa59653a021b None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:37.077776 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea39a6-990c-4d47-86d9-fa59653a021b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:37.077870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea39a6-990c-4d47-86d9-fa59653a021b None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:37.079174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 569/1147] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:37.084394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a09a3b-2c5d-4534-8d16-b23a7f4e00db None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:37.084762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a09a3b-2c5d-4534-8d16-b23a7f4e00db None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:37.084964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a09a3b-2c5d-4534-8d16-b23a7f4e00db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:37.102394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-75a09a3b-2c5d-4534-8d16-b23a7f4e00db None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:37.103576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-75a09a3b-2c5d-4534-8d16-b23a7f4e00db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:37.166857 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 579/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:37.189497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea39a6-990c-4d47-86d9-fa59653a021b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:37.269192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:37.289015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea39a6-990c-4d47-86d9-fa59653a021b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:37.322165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17169a-c0bb-442a-b8ad-0f5c98a5c0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:37.322943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17169a-c0bb-442a-b8ad-0f5c98a5c0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:37.323227 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17169a-c0bb-442a-b8ad-0f5c98a5c0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:37.323500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17169a-c0bb-442a-b8ad-0f5c98a5c0f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:37.333665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17169a-c0bb-442a-b8ad-0f5c98a5c0f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-dd17169a-c0bb-442a-b8ad-0f5c98a5c0f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:08:37.341725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:37.342796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:37.343977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 570/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:08:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:37.349075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a09a3b-2c5d-4534-8d16-b23a7f4e00db None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:37.382847 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:37.446092 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a09a3b-2c5d-4534-8d16-b23a7f4e00db None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:37.478849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257755c4-fca3-4a7e-9ab3-3591d4e67062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:37.479505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257755c4-fca3-4a7e-9ab3-3591d4e67062 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:37.479792 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257755c4-fca3-4a7e-9ab3-3591d4e67062 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:37.480095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257755c4-fca3-4a7e-9ab3-3591d4e67062 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:37.480839 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257755c4-fca3-4a7e-9ab3-3591d4e67062 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:37.481147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257755c4-fca3-4a7e-9ab3-3591d4e67062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:37.482493 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257755c4-fca3-4a7e-9ab3-3591d4e67062 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:37.490240 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 580/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:08:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:37.496385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17169a-c0bb-442a-b8ad-0f5c98a5c0f8 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:37.591824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17169a-c0bb-442a-b8ad-0f5c98a5c0f8 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:37.623980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa40f4-27ce-43f1-9dd5-2dca600c89d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:37.624854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa40f4-27ce-43f1-9dd5-2dca600c89d8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:37.625139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa40f4-27ce-43f1-9dd5-2dca600c89d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:37.625412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa40f4-27ce-43f1-9dd5-2dca600c89d8 None admin] PATH_INFO: `/v3/users/5bbb0aba6a394e67998daa1cef443659` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:37.626573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa40f4-27ce-43f1-9dd5-2dca600c89d8 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbb0aba6a394e67998daa1cef443659)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:37.627990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa40f4-27ce-43f1-9dd5-2dca600c89d8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:37.629599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 571/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:37 2022] GET /identity/v3/users/5bbb0aba6a394e67998daa1cef443659 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:37.635088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257755c4-fca3-4a7e-9ab3-3591d4e67062 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:37.728460 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:37.732471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257755c4-fca3-4a7e-9ab3-3591d4e67062 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:37.764739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6448d-e53e-4fdf-ad47-0bdb7f362a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:37.765575 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6448d-e53e-4fdf-ad47-0bdb7f362a83 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:37.765858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6448d-e53e-4fdf-ad47-0bdb7f362a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:37.766130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6448d-e53e-4fdf-ad47-0bdb7f362a83 None admin] PATH_INFO: `/v3/projects/4d9591df09ea465c80603f23837b92a3` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:37.767281 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6448d-e53e-4fdf-ad47-0bdb7f362a83 None admin] RBAC: Authorizing `identity:get_project(project_id=4d9591df09ea465c80603f23837b92a3)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:37.768719 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6448d-e53e-4fdf-ad47-0bdb7f362a83 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:37.770372 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 581/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:08:37 2022] GET /identity/v3/projects/4d9591df09ea465c80603f23837b92a3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:37.776852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa40f4-27ce-43f1-9dd5-2dca600c89d8 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:37.787688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:37.874709 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa40f4-27ce-43f1-9dd5-2dca600c89d8 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:37.907265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b2cf1-730c-4039-bb23-41b066427fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:37.908170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b2cf1-730c-4039-bb23-41b066427fef None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:37.908461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b2cf1-730c-4039-bb23-41b066427fef None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:37.908738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b2cf1-730c-4039-bb23-41b066427fef None admin] PATH_INFO: `/v3/projects/4d9591df09ea465c80603f23837b92a3/users/5bbb0aba6a394e67998daa1cef443659/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:37.910694 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b2cf1-730c-4039-bb23-41b066427fef None admin] RBAC: Authorizing `identity:create_grant(project_id=4d9591df09ea465c80603f23837b92a3, user_id=5bbb0aba6a394e67998daa1cef443659, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:37.912135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b2cf1-730c-4039-bb23-41b066427fef None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:37.925341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 572/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:08:37 2022] PUT /identity/v3/projects/4d9591df09ea465c80603f23837b92a3/users/5bbb0aba6a394e67998daa1cef443659/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:08:39.978668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03f43e-4810-4ca2-916f-41cfc754ab50 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:39.979232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03f43e-4810-4ca2-916f-41cfc754ab50 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:39.979232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03f43e-4810-4ca2-916f-41cfc754ab50 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:39.980668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 582/1154] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:39.985628 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240e58a-631a-4a51-8602-83757efa6ba4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:39.986040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240e58a-631a-4a51-8602-83757efa6ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:39.986200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240e58a-631a-4a51-8602-83757efa6ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:40.003147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-1240e58a-631a-4a51-8602-83757efa6ba4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:40.004364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1240e58a-631a-4a51-8602-83757efa6ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:40.063664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:40.070518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 573/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:40.093647 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03f43e-4810-4ca2-916f-41cfc754ab50 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:40.192592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03f43e-4810-4ca2-916f-41cfc754ab50 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:40.225448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932819e9-3950-4f19-b6ed-895c6631e8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:40.226807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932819e9-3950-4f19-b6ed-895c6631e8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:40.227096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932819e9-3950-4f19-b6ed-895c6631e8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:40.227378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932819e9-3950-4f19-b6ed-895c6631e8e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:40.237452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932819e9-3950-4f19-b6ed-895c6631e8e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-932819e9-3950-4f19-b6ed-895c6631e8e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:08:40.245105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:40.246194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:40.247246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 583/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:08:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:40.252436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240e58a-631a-4a51-8602-83757efa6ba4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:40.347363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240e58a-631a-4a51-8602-83757efa6ba4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:40.380857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65002636-b2e3-4648-9429-f19272773da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:40.381528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65002636-b2e3-4648-9429-f19272773da5 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:40.382329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65002636-b2e3-4648-9429-f19272773da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:40.382622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65002636-b2e3-4648-9429-f19272773da5 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:40.383369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65002636-b2e3-4648-9429-f19272773da5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:40.383682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65002636-b2e3-4648-9429-f19272773da5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:40.385102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65002636-b2e3-4648-9429-f19272773da5 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:40.393337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 574/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:08:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:40.399684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932819e9-3950-4f19-b6ed-895c6631e8e8 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:40.426599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:40.498330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932819e9-3950-4f19-b6ed-895c6631e8e8 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:40.530642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461bea9-c892-4463-aaf9-eb039f1027f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:40.531488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461bea9-c892-4463-aaf9-eb039f1027f8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:40.531777 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461bea9-c892-4463-aaf9-eb039f1027f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:40.532058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461bea9-c892-4463-aaf9-eb039f1027f8 None admin] PATH_INFO: `/v3/users/5bbb0aba6a394e67998daa1cef443659` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:40.533261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461bea9-c892-4463-aaf9-eb039f1027f8 None admin] RBAC: Authorizing `identity:get_user(user_id=5bbb0aba6a394e67998daa1cef443659)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:40.534687 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461bea9-c892-4463-aaf9-eb039f1027f8 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:40.536336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 584/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:40 2022] GET /identity/v3/users/5bbb0aba6a394e67998daa1cef443659 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:40.542345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65002636-b2e3-4648-9429-f19272773da5 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:40.598765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:40.639828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65002636-b2e3-4648-9429-f19272773da5 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:40.672551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21396ee-d7ad-4a1f-8f70-2a628c8136d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:40.673190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21396ee-d7ad-4a1f-8f70-2a628c8136d9 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:40.673479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21396ee-d7ad-4a1f-8f70-2a628c8136d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:40.673758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21396ee-d7ad-4a1f-8f70-2a628c8136d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:40.737781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21396ee-d7ad-4a1f-8f70-2a628c8136d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:40.739223 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21396ee-d7ad-4a1f-8f70-2a628c8136d9 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:40.794197 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:40.834314 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 575/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:08:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:08:40.840254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461bea9-c892-4463-aaf9-eb039f1027f8 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:40.929698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:40.936462 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461bea9-c892-4463-aaf9-eb039f1027f8 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:40.968367 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e3909-808f-4f41-8833-d6575fc5da57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:40.969204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e3909-808f-4f41-8833-d6575fc5da57 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:40.969489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e3909-808f-4f41-8833-d6575fc5da57 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:40.969765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e3909-808f-4f41-8833-d6575fc5da57 None admin] PATH_INFO: `/v3/projects/4d9591df09ea465c80603f23837b92a3` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:40.970922 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e3909-808f-4f41-8833-d6575fc5da57 None admin] RBAC: Authorizing `identity:get_project(project_id=4d9591df09ea465c80603f23837b92a3)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:40.972370 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e3909-808f-4f41-8833-d6575fc5da57 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:40.974036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 585/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:08:40 2022] GET /identity/v3/projects/4d9591df09ea465c80603f23837b92a3 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:40.979898 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21396ee-d7ad-4a1f-8f70-2a628c8136d9 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:41.075290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21396ee-d7ad-4a1f-8f70-2a628c8136d9 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:41.107326 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee938bf2-6ad2-4ec8-9cb6-681e56513210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:41.107964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee938bf2-6ad2-4ec8-9cb6-681e56513210 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:41.108280 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee938bf2-6ad2-4ec8-9cb6-681e56513210 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:41.108564 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee938bf2-6ad2-4ec8-9cb6-681e56513210 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:41.109384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee938bf2-6ad2-4ec8-9cb6-681e56513210 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=4d9591df09ea465c80603f23837b92a3, user.id=5bbb0aba6a394e67998daa1cef443659) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:41.109701 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee938bf2-6ad2-4ec8-9cb6-681e56513210 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:41.111113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee938bf2-6ad2-4ec8-9cb6-681e56513210 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:41.117333 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 576/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:08:40 2022] GET /identity/v3/role_assignments?user.id=5bbb0aba6a394e67998daa1cef443659&scope.project.id=4d9591df09ea465c80603f23837b92a3&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:43.223398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b6b2c-4a3f-4202-b29b-b2e7277cb92a None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:43.223953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b6b2c-4a3f-4202-b29b-b2e7277cb92a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:43.223953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b6b2c-4a3f-4202-b29b-b2e7277cb92a None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:43.225446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 586/1162] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:43.230769 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e535d55-84a5-4217-8953-abb638659eae None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:43.231132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e535d55-84a5-4217-8953-abb638659eae None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:43.231331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e535d55-84a5-4217-8953-abb638659eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:43.248832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-4e535d55-84a5-4217-8953-abb638659eae None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:43.250016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-4e535d55-84a5-4217-8953-abb638659eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:43.267076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:43.314213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 577/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:43.336375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b6b2c-4a3f-4202-b29b-b2e7277cb92a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:43.434581 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b6b2c-4a3f-4202-b29b-b2e7277cb92a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:43.467017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76348e87-b9fa-45f6-8ac0-c3a275404eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:43.467641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76348e87-b9fa-45f6-8ac0-c3a275404eaa None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:43.467925 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76348e87-b9fa-45f6-8ac0-c3a275404eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:43.468239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76348e87-b9fa-45f6-8ac0-c3a275404eaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:43.496319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:43.533658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76348e87-b9fa-45f6-8ac0-c3a275404eaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:43.535077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76348e87-b9fa-45f6-8ac0-c3a275404eaa None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:43.626754 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 587/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:08:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:08:43.633100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e535d55-84a5-4217-8953-abb638659eae None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:43.727342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e535d55-84a5-4217-8953-abb638659eae None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:43.760282 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33bba6-3df0-4295-beb6-f448b3ed03e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:43.761014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33bba6-3df0-4295-beb6-f448b3ed03e1 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:43.761299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33bba6-3df0-4295-beb6-f448b3ed03e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:43.761577 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33bba6-3df0-4295-beb6-f448b3ed03e1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:43.768030 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33bba6-3df0-4295-beb6-f448b3ed03e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:43.769506 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33bba6-3df0-4295-beb6-f448b3ed03e1 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-ef33bba6-3df0-4295-beb6-f448b3ed03e1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:43.783608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:43.784803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:43.785675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 578/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jun 17 07:08:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:43.790561 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76348e87-b9fa-45f6-8ac0-c3a275404eaa None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:43.848053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:43.884824 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76348e87-b9fa-45f6-8ac0-c3a275404eaa None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:43.917599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afa15d-0fa1-4d1b-8f08-dcf446f0fef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:43.918232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afa15d-0fa1-4d1b-8f08-dcf446f0fef7 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:43.918518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afa15d-0fa1-4d1b-8f08-dcf446f0fef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:43.918814 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afa15d-0fa1-4d1b-8f08-dcf446f0fef7 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:43.919517 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afa15d-0fa1-4d1b-8f08-dcf446f0fef7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:43.919822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afa15d-0fa1-4d1b-8f08-dcf446f0fef7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:43.921198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afa15d-0fa1-4d1b-8f08-dcf446f0fef7 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:43.928003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 588/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jun 17 07:08:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:45.994847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9431f-c122-4f3a-8827-a0eff5d041a4 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:45.995372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9431f-c122-4f3a-8827-a0eff5d041a4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:45.995489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9431f-c122-4f3a-8827-a0eff5d041a4 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:45.996815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 579/1167] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:46.001891 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94434b29-cfc4-4acb-a66a-b5e18b93c40f None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:46.002325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94434b29-cfc4-4acb-a66a-b5e18b93c40f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:46.002447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94434b29-cfc4-4acb-a66a-b5e18b93c40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:46.018569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-94434b29-cfc4-4acb-a66a-b5e18b93c40f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:46.019732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-94434b29-cfc4-4acb-a66a-b5e18b93c40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:46.082658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 589/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 07:08:46.105459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9431f-c122-4f3a-8827-a0eff5d041a4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:46.121679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:46.204527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9431f-c122-4f3a-8827-a0eff5d041a4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:46.236934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71def98-cf1f-481d-bff4-b8d03b303e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:46.237574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71def98-cf1f-481d-bff4-b8d03b303e5e None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:46.237861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71def98-cf1f-481d-bff4-b8d03b303e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:46.238139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71def98-cf1f-481d-bff4-b8d03b303e5e None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:46.238862 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71def98-cf1f-481d-bff4-b8d03b303e5e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:46.240251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71def98-cf1f-481d-bff4-b8d03b303e5e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:46.255110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 580/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Fri Jun 17 07:08:46 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:08:48.345893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0966c-6741-406a-9f88-953f2d0a7790 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:48.346423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0966c-6741-406a-9f88-953f2d0a7790 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:48.346423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0966c-6741-406a-9f88-953f2d0a7790 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:48.347814 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 590/1170] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:08:48.353526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88399636-44cb-45a4-a7e0-5c9d826587b5 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:48.353944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88399636-44cb-45a4-a7e0-5c9d826587b5 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:48.354087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88399636-44cb-45a4-a7e0-5c9d826587b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:48.370338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-88399636-44cb-45a4-a7e0-5c9d826587b5 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:48.371524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-88399636-44cb-45a4-a7e0-5c9d826587b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:48.435513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 581/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:48.458350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0966c-6741-406a-9f88-953f2d0a7790 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:48.513436 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:48.558714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0966c-6741-406a-9f88-953f2d0a7790 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:48.591042 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad43c2-f4f9-40d5-a60d-942e9721b091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:48.591887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad43c2-f4f9-40d5-a60d-942e9721b091 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:48.592204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad43c2-f4f9-40d5-a60d-942e9721b091 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:48.592482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad43c2-f4f9-40d5-a60d-942e9721b091 None admin] PATH_INFO: `/v3/domains/59a11215c0864835a849ea3d54df293e` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:48.602535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad43c2-f4f9-40d5-a60d-942e9721b091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=59a11215c0864835a849ea3d54df293e)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:48.603917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad43c2-f4f9-40d5-a60d-942e9721b091 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:48.605576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 591/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Fri Jun 17 07:08:48 2022] GET /identity/v3/domains/59a11215c0864835a849ea3d54df293e => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:48.611361 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88399636-44cb-45a4-a7e0-5c9d826587b5 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:48.614396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:48.706530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88399636-44cb-45a4-a7e0-5c9d826587b5 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:48.738852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42a719-8ebe-48ed-beaf-afb44515fe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:48.739498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42a719-8ebe-48ed-beaf-afb44515fe6f None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:48.739780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42a719-8ebe-48ed-beaf-afb44515fe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:48.740054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42a719-8ebe-48ed-beaf-afb44515fe6f None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:48.740785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42a719-8ebe-48ed-beaf-afb44515fe6f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:48.742757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42a719-8ebe-48ed-beaf-afb44515fe6f None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:48.766152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 582/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:08:48 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:08:50.886474 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a0688-406b-45aa-9768-87f1c0650e57 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:50.887057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a0688-406b-45aa-9768-87f1c0650e57 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:50.887057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a0688-406b-45aa-9768-87f1c0650e57 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:50.888408 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 592/1174] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:50.893569 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eba277-67ec-437f-a9e5-bdf20cb1c801 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:50.894012 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eba277-67ec-437f-a9e5-bdf20cb1c801 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:50.894124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eba277-67ec-437f-a9e5-bdf20cb1c801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:50.911137 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-00eba277-67ec-437f-a9e5-bdf20cb1c801 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:50.912332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-00eba277-67ec-437f-a9e5-bdf20cb1c801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:50.961332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:50.977646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 583/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:51.001635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a0688-406b-45aa-9768-87f1c0650e57 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:51.101062 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a0688-406b-45aa-9768-87f1c0650e57 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:51.116616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:51.134366 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6173ed1-af85-4c1a-81a6-ecbab16ce422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:51.135215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6173ed1-af85-4c1a-81a6-ecbab16ce422 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:51.135499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6173ed1-af85-4c1a-81a6-ecbab16ce422 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:51.135772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6173ed1-af85-4c1a-81a6-ecbab16ce422 None admin] PATH_INFO: `/v3/domains/59a11215c0864835a849ea3d54df293e` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:51.137014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6173ed1-af85-4c1a-81a6-ecbab16ce422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=59a11215c0864835a849ea3d54df293e)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:51.138384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6173ed1-af85-4c1a-81a6-ecbab16ce422 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:51.139967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 593/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Fri Jun 17 07:08:51 2022] GET /identity/v3/domains/59a11215c0864835a849ea3d54df293e => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:51.145557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eba277-67ec-437f-a9e5-bdf20cb1c801 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:51.239197 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eba277-67ec-437f-a9e5-bdf20cb1c801 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:51.271980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44a2e1-20dc-4c1a-8b78-aefd643734c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:51.272697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44a2e1-20dc-4c1a-8b78-aefd643734c9 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:51.272983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44a2e1-20dc-4c1a-8b78-aefd643734c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:51.273262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44a2e1-20dc-4c1a-8b78-aefd643734c9 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:51.273964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44a2e1-20dc-4c1a-8b78-aefd643734c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:51.275652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44a2e1-20dc-4c1a-8b78-aefd643734c9 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:51.295611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 584/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:08:51 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:08:53.414867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c0fe0-b17d-49b9-acdb-e48a55e37b95 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:53.415345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c0fe0-b17d-49b9-acdb-e48a55e37b95 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:53.415445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c0fe0-b17d-49b9-acdb-e48a55e37b95 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:53.416812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 594/1178] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:53.421914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ef169-acaa-455f-8dac-7cae0df2f443 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:53.422326 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ef169-acaa-455f-8dac-7cae0df2f443 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:53.422478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ef169-acaa-455f-8dac-7cae0df2f443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:53.439268 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-1c8ef169-acaa-455f-8dac-7cae0df2f443 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:53.440484 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1c8ef169-acaa-455f-8dac-7cae0df2f443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:53.473521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:53.506232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 585/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:53.529984 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c0fe0-b17d-49b9-acdb-e48a55e37b95 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:53.627821 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c0fe0-b17d-49b9-acdb-e48a55e37b95 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:53.660716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9500990-5b5d-44a1-b1a7-0759372f00ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:53.661430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9500990-5b5d-44a1-b1a7-0759372f00ac None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:53.661715 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9500990-5b5d-44a1-b1a7-0759372f00ac None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:53.661990 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9500990-5b5d-44a1-b1a7-0759372f00ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:53.671800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9500990-5b5d-44a1-b1a7-0759372f00ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-f9500990-5b5d-44a1-b1a7-0759372f00ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:08:53.679936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:53.684148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:53.685362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 595/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:08:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:53.687530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ef169-acaa-455f-8dac-7cae0df2f443 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:53.782292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ef169-acaa-455f-8dac-7cae0df2f443 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:53.815087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b3dc1-6e3f-401d-a71f-d633ee29282d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:53.815744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b3dc1-6e3f-401d-a71f-d633ee29282d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:53.816031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b3dc1-6e3f-401d-a71f-d633ee29282d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:53.816347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b3dc1-6e3f-401d-a71f-d633ee29282d None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:53.817087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b3dc1-6e3f-401d-a71f-d633ee29282d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:53.817397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b3dc1-6e3f-401d-a71f-d633ee29282d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:53.818765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b3dc1-6e3f-401d-a71f-d633ee29282d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:53.826581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 586/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:08:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:53.832305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9500990-5b5d-44a1-b1a7-0759372f00ac None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:53.876459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:53.928545 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9500990-5b5d-44a1-b1a7-0759372f00ac None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:53.960324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d4449-5e3a-4cfd-b10f-5354623be00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:53.961172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d4449-5e3a-4cfd-b10f-5354623be00b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:53.961458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d4449-5e3a-4cfd-b10f-5354623be00b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:53.961738 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d4449-5e3a-4cfd-b10f-5354623be00b None admin] PATH_INFO: `/v3/users/2d15f475c98749269d81ba4382c0d687` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:53.974169 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d4449-5e3a-4cfd-b10f-5354623be00b None admin] RBAC: Authorizing `identity:get_user(user_id=2d15f475c98749269d81ba4382c0d687)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:53.976159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d4449-5e3a-4cfd-b10f-5354623be00b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:53.977768 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 596/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:53 2022] GET /identity/v3/users/2d15f475c98749269d81ba4382c0d687 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:53.983257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b3dc1-6e3f-401d-a71f-d633ee29282d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:54.016483 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:54.079907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b3dc1-6e3f-401d-a71f-d633ee29282d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:54.119476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eccc64-5324-4fcd-9e7b-f731a53b41ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:54.120839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eccc64-5324-4fcd-9e7b-f731a53b41ce None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:54.121134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eccc64-5324-4fcd-9e7b-f731a53b41ce None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:54.121417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eccc64-5324-4fcd-9e7b-f731a53b41ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:54.186581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eccc64-5324-4fcd-9e7b-f731a53b41ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:54.188033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eccc64-5324-4fcd-9e7b-f731a53b41ce None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:54.218832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:54.283318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 587/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:08:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:08:54.289197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d4449-5e3a-4cfd-b10f-5354623be00b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:54.384726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d4449-5e3a-4cfd-b10f-5354623be00b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:54.386194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:54.419949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89820c9d-0107-47e5-97d0-c1ccac4fd36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:54.420838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89820c9d-0107-47e5-97d0-c1ccac4fd36c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:54.421126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89820c9d-0107-47e5-97d0-c1ccac4fd36c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:54.421403 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89820c9d-0107-47e5-97d0-c1ccac4fd36c None admin] PATH_INFO: `/v3/projects/230c7ac8434c4d71a66fb4a4f3258187` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:54.422573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89820c9d-0107-47e5-97d0-c1ccac4fd36c None admin] RBAC: Authorizing `identity:get_project(project_id=230c7ac8434c4d71a66fb4a4f3258187)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:54.424002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89820c9d-0107-47e5-97d0-c1ccac4fd36c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:54.425726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 597/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:08:54 2022] GET /identity/v3/projects/230c7ac8434c4d71a66fb4a4f3258187 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:54.431654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eccc64-5324-4fcd-9e7b-f731a53b41ce None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:54.527667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eccc64-5324-4fcd-9e7b-f731a53b41ce None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:54.559971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a965ae8-3f0e-4b13-b425-64c158b5e900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:54.560643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a965ae8-3f0e-4b13-b425-64c158b5e900 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:54.560930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a965ae8-3f0e-4b13-b425-64c158b5e900 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:54.561210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a965ae8-3f0e-4b13-b425-64c158b5e900 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:54.562032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a965ae8-3f0e-4b13-b425-64c158b5e900 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=230c7ac8434c4d71a66fb4a4f3258187, user.id=2d15f475c98749269d81ba4382c0d687) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:54.562347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a965ae8-3f0e-4b13-b425-64c158b5e900 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:54.563773 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a965ae8-3f0e-4b13-b425-64c158b5e900 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:54.565610 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:54.570378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 588/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:08:54 2022] GET /identity/v3/role_assignments?user.id=2d15f475c98749269d81ba4382c0d687&scope.project.id=230c7ac8434c4d71a66fb4a4f3258187&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:56.605496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f595f-705d-4891-953a-edbccecb3329 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:56.606058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f595f-705d-4891-953a-edbccecb3329 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:56.606058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f595f-705d-4891-953a-edbccecb3329 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:56.607401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 598/1186] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:56.613679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa404c-1988-4a4a-bb28-00f99b862315 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:56.614342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa404c-1988-4a4a-bb28-00f99b862315 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:56.614342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa404c-1988-4a4a-bb28-00f99b862315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:56.630994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-3afa404c-1988-4a4a-bb28-00f99b862315 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:56.632768 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3afa404c-1988-4a4a-bb28-00f99b862315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:56.696826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 589/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:56.719151 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f595f-705d-4891-953a-edbccecb3329 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:56.816288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f595f-705d-4891-953a-edbccecb3329 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:56.848502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226473e3-d660-4556-8ebd-552059f7fbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:56.849272 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226473e3-d660-4556-8ebd-552059f7fbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:56.849557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226473e3-d660-4556-8ebd-552059f7fbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:56.849833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226473e3-d660-4556-8ebd-552059f7fbf0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:56.859528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226473e3-d660-4556-8ebd-552059f7fbf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-226473e3-d660-4556-8ebd-552059f7fbf0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:08:56.867649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:56.868649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:08:56.869502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 599/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:08:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:56.874626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa404c-1988-4a4a-bb28-00f99b862315 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:56.970257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa404c-1988-4a4a-bb28-00f99b862315 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:56.995970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:57.003866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a4b14-d4df-4359-96c8-98e9e211ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:57.004568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a4b14-d4df-4359-96c8-98e9e211ce1a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:57.004857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a4b14-d4df-4359-96c8-98e9e211ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:57.005138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a4b14-d4df-4359-96c8-98e9e211ce1a None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:57.005877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a4b14-d4df-4359-96c8-98e9e211ce1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:57.006189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a4b14-d4df-4359-96c8-98e9e211ce1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:57.007585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a4b14-d4df-4359-96c8-98e9e211ce1a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:57.015399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 590/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:08:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:57.020964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226473e3-d660-4556-8ebd-552059f7fbf0 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:57.051051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:57.118059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226473e3-d660-4556-8ebd-552059f7fbf0 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:57.150957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be2cf8-f911-445a-bf50-f077c05535c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:57.151801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be2cf8-f911-445a-bf50-f077c05535c6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:57.152110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be2cf8-f911-445a-bf50-f077c05535c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:57.152393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be2cf8-f911-445a-bf50-f077c05535c6 None admin] PATH_INFO: `/v3/users/2d15f475c98749269d81ba4382c0d687` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:57.153561 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be2cf8-f911-445a-bf50-f077c05535c6 None admin] RBAC: Authorizing `identity:get_user(user_id=2d15f475c98749269d81ba4382c0d687)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:57.154980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be2cf8-f911-445a-bf50-f077c05535c6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:57.156672 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 600/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:57 2022] GET /identity/v3/users/2d15f475c98749269d81ba4382c0d687 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:57.162289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a4b14-d4df-4359-96c8-98e9e211ce1a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:57.259712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a4b14-d4df-4359-96c8-98e9e211ce1a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:57.291755 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727568b3-86f5-4b93-8e61-1e0b76e0c38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:57.292622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727568b3-86f5-4b93-8e61-1e0b76e0c38a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:57.292902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727568b3-86f5-4b93-8e61-1e0b76e0c38a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:57.293174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727568b3-86f5-4b93-8e61-1e0b76e0c38a None admin] PATH_INFO: `/v3/projects/230c7ac8434c4d71a66fb4a4f3258187` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:57.294328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727568b3-86f5-4b93-8e61-1e0b76e0c38a None admin] RBAC: Authorizing `identity:get_project(project_id=230c7ac8434c4d71a66fb4a4f3258187)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:57.295744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727568b3-86f5-4b93-8e61-1e0b76e0c38a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:57.297434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 591/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:08:57 2022] GET /identity/v3/projects/230c7ac8434c4d71a66fb4a4f3258187 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:57.303726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be2cf8-f911-445a-bf50-f077c05535c6 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:57.396207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:57.400299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be2cf8-f911-445a-bf50-f077c05535c6 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:57.434598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59273d30-e3ad-43d1-b2ef-d23e5206dff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:57.435503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59273d30-e3ad-43d1-b2ef-d23e5206dff0 None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:57.435790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59273d30-e3ad-43d1-b2ef-d23e5206dff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:57.436090 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59273d30-e3ad-43d1-b2ef-d23e5206dff0 None admin] PATH_INFO: `/v3/projects/230c7ac8434c4d71a66fb4a4f3258187/users/2d15f475c98749269d81ba4382c0d687/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:57.438066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59273d30-e3ad-43d1-b2ef-d23e5206dff0 None admin] RBAC: Authorizing `identity:create_grant(project_id=230c7ac8434c4d71a66fb4a4f3258187, user_id=2d15f475c98749269d81ba4382c0d687, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:57.439498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59273d30-e3ad-43d1-b2ef-d23e5206dff0 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:57.452801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 601/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:08:57 2022] PUT /identity/v3/projects/230c7ac8434c4d71a66fb4a4f3258187/users/2d15f475c98749269d81ba4382c0d687/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:08:59.490265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2a45a-441c-4a9b-85bd-045da2cd7416 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:59.490814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2a45a-441c-4a9b-85bd-045da2cd7416 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:59.490929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2a45a-441c-4a9b-85bd-045da2cd7416 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:59.492275 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 592/1193] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:08:59.497385 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18606b7-061e-4aad-8121-88bcb73a4c5f None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:59.497812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18606b7-061e-4aad-8121-88bcb73a4c5f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:59.497948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18606b7-061e-4aad-8121-88bcb73a4c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:59.514147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c18606b7-061e-4aad-8121-88bcb73a4c5f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:08:59.515317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c18606b7-061e-4aad-8121-88bcb73a4c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:08:59.580993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 602/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:08:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:08:59.603574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2a45a-441c-4a9b-85bd-045da2cd7416 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:59.661553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:59.702808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2a45a-441c-4a9b-85bd-045da2cd7416 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:59.734505 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce55cbe-264e-4913-906c-8c2a0b61fd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:59.735277 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce55cbe-264e-4913-906c-8c2a0b61fd19 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:59.735562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce55cbe-264e-4913-906c-8c2a0b61fd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:59.735846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce55cbe-264e-4913-906c-8c2a0b61fd19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:59.746131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce55cbe-264e-4913-906c-8c2a0b61fd19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-5ce55cbe-264e-4913-906c-8c2a0b61fd19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:08:59.753721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:08:59.754881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:08:59.756045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 593/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:08:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:08:59.760779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18606b7-061e-4aad-8121-88bcb73a4c5f None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:08:59.844540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:08:59.857199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18606b7-061e-4aad-8121-88bcb73a4c5f None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:08:59.889747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d78d8c-b3ab-450f-9011-abb5687bb9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:08:59.890401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d78d8c-b3ab-450f-9011-abb5687bb9ad None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:08:59.890690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d78d8c-b3ab-450f-9011-abb5687bb9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:08:59.890968 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d78d8c-b3ab-450f-9011-abb5687bb9ad None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:08:59.891730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d78d8c-b3ab-450f-9011-abb5687bb9ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:08:59.892041 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d78d8c-b3ab-450f-9011-abb5687bb9ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:08:59.893436 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d78d8c-b3ab-450f-9011-abb5687bb9ad None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:08:59.902171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 603/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:08:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:08:59.907763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce55cbe-264e-4913-906c-8c2a0b61fd19 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:09:00.002178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce55cbe-264e-4913-906c-8c2a0b61fd19 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:09:00.018288 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:09:00.035003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c1bba-218a-4344-b1e9-221be1f16602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:09:00.035850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c1bba-218a-4344-b1e9-221be1f16602 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:00.036157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c1bba-218a-4344-b1e9-221be1f16602 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:00.036433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c1bba-218a-4344-b1e9-221be1f16602 None admin] PATH_INFO: `/v3/users/2d15f475c98749269d81ba4382c0d687` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:00.037623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c1bba-218a-4344-b1e9-221be1f16602 None admin] RBAC: Authorizing `identity:get_user(user_id=2d15f475c98749269d81ba4382c0d687)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:09:00.039044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c1bba-218a-4344-b1e9-221be1f16602 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:09:00.040673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 594/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jun 17 07:08:59 2022] GET /identity/v3/users/2d15f475c98749269d81ba4382c0d687 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:09:00.046206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d78d8c-b3ab-450f-9011-abb5687bb9ad None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:09:00.141212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d78d8c-b3ab-450f-9011-abb5687bb9ad None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:09:00.173457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed715c-e972-4de0-a525-61aba246de9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:09:00.174087 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed715c-e972-4de0-a525-61aba246de9d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:00.174375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed715c-e972-4de0-a525-61aba246de9d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:00.174656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed715c-e972-4de0-a525-61aba246de9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:00.192131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:09:00.239345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed715c-e972-4de0-a525-61aba246de9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:09:00.241319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed715c-e972-4de0-a525-61aba246de9d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:09:00.335423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 604/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:09:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:09:00.341273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c1bba-218a-4344-b1e9-221be1f16602 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:09:00.437810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c1bba-218a-4344-b1e9-221be1f16602 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:09:00.470773 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0456d-ff47-4078-b8c3-132b7a1fb649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:09:00.471623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0456d-ff47-4078-b8c3-132b7a1fb649 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:00.471908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0456d-ff47-4078-b8c3-132b7a1fb649 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:00.472215 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0456d-ff47-4078-b8c3-132b7a1fb649 None admin] PATH_INFO: `/v3/projects/230c7ac8434c4d71a66fb4a4f3258187` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:00.473392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0456d-ff47-4078-b8c3-132b7a1fb649 None admin] RBAC: Authorizing `identity:get_project(project_id=230c7ac8434c4d71a66fb4a4f3258187)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:09:00.474824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0456d-ff47-4078-b8c3-132b7a1fb649 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:09:00.477080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 595/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jun 17 07:09:00 2022] GET /identity/v3/projects/230c7ac8434c4d71a66fb4a4f3258187 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:09:00.483206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed715c-e972-4de0-a525-61aba246de9d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:09:00.532495 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:09:00.583723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed715c-e972-4de0-a525-61aba246de9d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:09:00.620539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f0192-3b75-4300-8ae5-ca050ebf6514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:09:00.621117 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f0192-3b75-4300-8ae5-ca050ebf6514 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:00.621403 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f0192-3b75-4300-8ae5-ca050ebf6514 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:00.621681 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f0192-3b75-4300-8ae5-ca050ebf6514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:00.622499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f0192-3b75-4300-8ae5-ca050ebf6514 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=230c7ac8434c4d71a66fb4a4f3258187, user.id=2d15f475c98749269d81ba4382c0d687) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:09:00.622813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f0192-3b75-4300-8ae5-ca050ebf6514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:09:00.624256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f0192-3b75-4300-8ae5-ca050ebf6514 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:09:00.630142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 605/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:09:00 2022] GET /identity/v3/role_assignments?user.id=2d15f475c98749269d81ba4382c0d687&scope.project.id=230c7ac8434c4d71a66fb4a4f3258187&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:09:10.466639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=73 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:09:10.627305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=43 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:09:55.960770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ec0a8-934f-444a-b4b3-fa16bfa958d5 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:55.961347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ec0a8-934f-444a-b4b3-fa16bfa958d5 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:55.961538 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ec0a8-934f-444a-b4b3-fa16bfa958d5 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:55.963211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 596/1201] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:09:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:09:55.968864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd265a40-91af-44da-8bd7-2b0a5b0ff846 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:55.969443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd265a40-91af-44da-8bd7-2b0a5b0ff846 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:55.969443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd265a40-91af-44da-8bd7-2b0a5b0ff846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:55.988661 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-bd265a40-91af-44da-8bd7-2b0a5b0ff846 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:09:55.990695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-bd265a40-91af-44da-8bd7-2b0a5b0ff846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:09:56.088212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 606/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:09:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:09:56.109198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ec0a8-934f-444a-b4b3-fa16bfa958d5 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:09:56.252842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ec0a8-934f-444a-b4b3-fa16bfa958d5 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:09:56.297941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925506e-fa70-4f59-8f0a-239024da0a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:09:56.299082 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925506e-fa70-4f59-8f0a-239024da0a65 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:56.300102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925506e-fa70-4f59-8f0a-239024da0a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:56.300102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925506e-fa70-4f59-8f0a-239024da0a65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:56.314749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925506e-fa70-4f59-8f0a-239024da0a65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:09:56.316327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925506e-fa70-4f59-8f0a-239024da0a65 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-5925506e-fa70-4f59-8f0a-239024da0a65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:09:56.340226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:09:56.341561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:09:56.342536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:09:56.342536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:09:56.342536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:09:56.342536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:09:56.342536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 597/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:09:56.348436 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd265a40-91af-44da-8bd7-2b0a5b0ff846 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:09:56.494845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd265a40-91af-44da-8bd7-2b0a5b0ff846 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:09:56.533017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:09:56.542038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab28da-12a6-4b41-869b-adcfc18fab46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:09:56.543310 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab28da-12a6-4b41-869b-adcfc18fab46 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:56.543745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab28da-12a6-4b41-869b-adcfc18fab46 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:56.544312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab28da-12a6-4b41-869b-adcfc18fab46 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:56.545166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab28da-12a6-4b41-869b-adcfc18fab46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:09:56.545812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab28da-12a6-4b41-869b-adcfc18fab46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:09:56.547736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab28da-12a6-4b41-869b-adcfc18fab46 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:09:56.563926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 607/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:09:56.571515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925506e-fa70-4f59-8f0a-239024da0a65 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:09:56.643157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:09:56.706060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925506e-fa70-4f59-8f0a-239024da0a65 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:09:56.748612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bd08b-f742-4273-8337-d21105f511ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:09:56.749308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bd08b-f742-4273-8337-d21105f511ef None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:56.749664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bd08b-f742-4273-8337-d21105f511ef None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:56.749982 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bd08b-f742-4273-8337-d21105f511ef None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:56.750724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bd08b-f742-4273-8337-d21105f511ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:09:56.752284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bd08b-f742-4273-8337-d21105f511ef None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:09:56.775313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 598/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jun 17 07:09:56 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:09:59.046234 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1455fd7-2d25-4928-b9fd-031d9144cec1 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:59.046798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1455fd7-2d25-4928-b9fd-031d9144cec1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:59.046798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1455fd7-2d25-4928-b9fd-031d9144cec1 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:59.048203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 608/1206] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:09:59.053852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdda17a-aabd-4584-ad76-07bbd6d8c4b0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:59.054264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdda17a-aabd-4584-ad76-07bbd6d8c4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:59.054470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdda17a-aabd-4584-ad76-07bbd6d8c4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:59.073351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6fdda17a-aabd-4584-ad76-07bbd6d8c4b0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:09:59.077200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6fdda17a-aabd-4584-ad76-07bbd6d8c4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:09:59.148489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 599/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:09:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:09:59.174608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1455fd7-2d25-4928-b9fd-031d9144cec1 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:09:59.296354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1455fd7-2d25-4928-b9fd-031d9144cec1 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:09:59.335351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a45cc5-dd6d-4deb-b5f0-5b0cbd3390de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:09:59.336270 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a45cc5-dd6d-4deb-b5f0-5b0cbd3390de None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:59.336623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a45cc5-dd6d-4deb-b5f0-5b0cbd3390de None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:59.336967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a45cc5-dd6d-4deb-b5f0-5b0cbd3390de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:59.349059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a45cc5-dd6d-4deb-b5f0-5b0cbd3390de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-21a45cc5-dd6d-4deb-b5f0-5b0cbd3390de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:09:59.357941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:09:59.359079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:09:59.359960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 609/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:09:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:09:59.366136 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdda17a-aabd-4584-ad76-07bbd6d8c4b0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:09:59.417958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:09:59.489215 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdda17a-aabd-4584-ad76-07bbd6d8c4b0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:09:59.532601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a256c25-1e2d-4456-9fb6-0828dafa9b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:09:59.533287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a256c25-1e2d-4456-9fb6-0828dafa9b07 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:59.533559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a256c25-1e2d-4456-9fb6-0828dafa9b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:59.533838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a256c25-1e2d-4456-9fb6-0828dafa9b07 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:59.534589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a256c25-1e2d-4456-9fb6-0828dafa9b07 None admin] RBAC: Adding query filter params (name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:09:59.534908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a256c25-1e2d-4456-9fb6-0828dafa9b07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:09:59.536304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a256c25-1e2d-4456-9fb6-0828dafa9b07 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:09:59.545150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 600/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:09:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:09:59.551390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a45cc5-dd6d-4deb-b5f0-5b0cbd3390de None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:09:59.616445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:09:59.687952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a45cc5-dd6d-4deb-b5f0-5b0cbd3390de None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:09:59.730744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d983-a245-40ef-a792-399541bb6b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:09:59.731589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d983-a245-40ef-a792-399541bb6b68 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:59.731899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d983-a245-40ef-a792-399541bb6b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:59.732212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d983-a245-40ef-a792-399541bb6b68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:59.746851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8d983-a245-40ef-a792-399541bb6b68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:09:59.748323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8d983-a245-40ef-a792-399541bb6b68 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-c1c8d983-a245-40ef-a792-399541bb6b68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:09:59.768576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:09:59.769597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:09:59.770532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:09:59.770532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:09:59.770532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:09:59.770532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:09:59.770532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 610/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:09:59.776762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a256c25-1e2d-4456-9fb6-0828dafa9b07 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:09:59.887719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:09:59.891289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a256c25-1e2d-4456-9fb6-0828dafa9b07 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:09:59.937750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40ca8e-59d7-4784-80b8-2aaa0662886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:09:59.938445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40ca8e-59d7-4784-80b8-2aaa0662886e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:09:59.938731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40ca8e-59d7-4784-80b8-2aaa0662886e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:09:59.939005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40ca8e-59d7-4784-80b8-2aaa0662886e None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:09:59.939741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40ca8e-59d7-4784-80b8-2aaa0662886e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:09:59.940217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40ca8e-59d7-4784-80b8-2aaa0662886e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:09:59.941650 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40ca8e-59d7-4784-80b8-2aaa0662886e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:09:59.957608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 601/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:09:59.963644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d983-a245-40ef-a792-399541bb6b68 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:00.086599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d983-a245-40ef-a792-399541bb6b68 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:00.088307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:00.128660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63895e-12f4-48d5-b658-345496bc1736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:00.129616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63895e-12f4-48d5-b658-345496bc1736 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:00.129955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63895e-12f4-48d5-b658-345496bc1736 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:00.130256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63895e-12f4-48d5-b658-345496bc1736 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:00.139817 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63895e-12f4-48d5-b658-345496bc1736 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:00.141604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63895e-12f4-48d5-b658-345496bc1736 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-df63895e-12f4-48d5-b658-345496bc1736 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:10:00.158440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:00.159688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:00.161253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:00.161253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:00.161253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:10:00.161253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:10:00.161253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:10:00.161253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:10:00.161253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:10:00.161253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:10:00.161253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 07:10:00.161253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:00.161253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 611/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:09:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:00.166094 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40ca8e-59d7-4784-80b8-2aaa0662886e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:00.292700 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40ca8e-59d7-4784-80b8-2aaa0662886e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:00.332477 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2d38e-747e-4da2-9db1-96a9d7b8960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:00.333261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2d38e-747e-4da2-9db1-96a9d7b8960d None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:00.334060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2d38e-747e-4da2-9db1-96a9d7b8960d None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:00.334375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2d38e-747e-4da2-9db1-96a9d7b8960d None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:00.335398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2d38e-747e-4da2-9db1-96a9d7b8960d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:00.335738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2d38e-747e-4da2-9db1-96a9d7b8960d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:00.337386 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2d38e-747e-4da2-9db1-96a9d7b8960d None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:00.361621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 602/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jun 17 07:10:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:00.362907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63895e-12f4-48d5-b658-345496bc1736 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:00.485770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63895e-12f4-48d5-b658-345496bc1736 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:00.526237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd20581-8e03-47aa-947d-b387bb580311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:00.527127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd20581-8e03-47aa-947d-b387bb580311 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:00.527417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd20581-8e03-47aa-947d-b387bb580311 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:00.527696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd20581-8e03-47aa-947d-b387bb580311 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:00.559205 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:00.602756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd20581-8e03-47aa-947d-b387bb580311 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:00.604237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd20581-8e03-47aa-947d-b387bb580311 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:00.706476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 612/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:10:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:10:00.712059 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2d38e-747e-4da2-9db1-96a9d7b8960d None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:00.732304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:00.837278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2d38e-747e-4da2-9db1-96a9d7b8960d None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:00.881508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bcb01-281e-40f5-9afa-536c7f1038f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:00.882438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bcb01-281e-40f5-9afa-536c7f1038f6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:00.882713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bcb01-281e-40f5-9afa-536c7f1038f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:00.882995 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bcb01-281e-40f5-9afa-536c7f1038f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:00.895403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bcb01-281e-40f5-9afa-536c7f1038f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:00.897529 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bcb01-281e-40f5-9afa-536c7f1038f6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-2f2bcb01-281e-40f5-9afa-536c7f1038f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:10:00.921209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:00.922189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:00.931124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:10:00.931124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:10:00.931124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:00.931124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:00.931124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 603/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:00.931477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd20581-8e03-47aa-947d-b387bb580311 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:00.998928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:01.042943 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd20581-8e03-47aa-947d-b387bb580311 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:01.083209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04edc6-e633-46aa-b37d-8b7a8ac64578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:01.084480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04edc6-e633-46aa-b37d-8b7a8ac64578 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:01.084767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04edc6-e633-46aa-b37d-8b7a8ac64578 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:01.085041 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04edc6-e633-46aa-b37d-8b7a8ac64578 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:01.085823 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04edc6-e633-46aa-b37d-8b7a8ac64578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:01.090365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04edc6-e633-46aa-b37d-8b7a8ac64578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:01.091785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04edc6-e633-46aa-b37d-8b7a8ac64578 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:01.111817 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 613/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:01.114181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bcb01-281e-40f5-9afa-536c7f1038f6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:01.216581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:01.254101 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bcb01-281e-40f5-9afa-536c7f1038f6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:01.291890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05ce5f-2ef0-426c-b399-b8e520f308c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:01.292869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05ce5f-2ef0-426c-b399-b8e520f308c6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:01.293219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05ce5f-2ef0-426c-b399-b8e520f308c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:01.293518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05ce5f-2ef0-426c-b399-b8e520f308c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:01.300355 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05ce5f-2ef0-426c-b399-b8e520f308c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:01.301959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05ce5f-2ef0-426c-b399-b8e520f308c6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-2c05ce5f-2ef0-426c-b399-b8e520f308c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:10:01.316567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:01.317691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:01.319201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:10:01.319201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:10:01.319201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:10:01.319201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:10:01.319201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:10:01.319201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:10:01.319201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:10:01.319201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:01.323054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 604/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:10:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:01.324441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04edc6-e633-46aa-b37d-8b7a8ac64578 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:01.444839 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04edc6-e633-46aa-b37d-8b7a8ac64578 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:01.477477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:01.491447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d73423-64dd-4fc7-963f-c5bff040296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:01.492144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d73423-64dd-4fc7-963f-c5bff040296c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:01.492435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d73423-64dd-4fc7-963f-c5bff040296c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:01.492714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d73423-64dd-4fc7-963f-c5bff040296c None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:01.493501 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d73423-64dd-4fc7-963f-c5bff040296c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:01.493814 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d73423-64dd-4fc7-963f-c5bff040296c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:01.495229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d73423-64dd-4fc7-963f-c5bff040296c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:01.508322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 614/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:10:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:01.514213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05ce5f-2ef0-426c-b399-b8e520f308c6 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:01.632442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05ce5f-2ef0-426c-b399-b8e520f308c6 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:01.667335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:01.674379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e88b44-7326-4402-8ffe-ec5ab70a3c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:01.675051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e88b44-7326-4402-8ffe-ec5ab70a3c96 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:01.675340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e88b44-7326-4402-8ffe-ec5ab70a3c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:01.675620 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e88b44-7326-4402-8ffe-ec5ab70a3c96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:01.676501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e88b44-7326-4402-8ffe-ec5ab70a3c96 None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=b8cf02a52d354517b363c0aaf477550c) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:01.676828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e88b44-7326-4402-8ffe-ec5ab70a3c96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:01.678840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e88b44-7326-4402-8ffe-ec5ab70a3c96 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:01.688264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 605/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:10:01 2022] GET /identity/v3/role_assignments?user.id=b8cf02a52d354517b363c0aaf477550c&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:03.847044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5933b57-ec21-4062-9c4e-61792da6051c None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:03.847693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5933b57-ec21-4062-9c4e-61792da6051c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:03.847693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5933b57-ec21-4062-9c4e-61792da6051c None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:03.849036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 615/1220] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:10:03.854540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe59bff-40f8-4779-975e-c4e51107ac88 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:03.854986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe59bff-40f8-4779-975e-c4e51107ac88 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:03.855096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe59bff-40f8-4779-975e-c4e51107ac88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:03.872215 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-dfe59bff-40f8-4779-975e-c4e51107ac88 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:10:03.873392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-dfe59bff-40f8-4779-975e-c4e51107ac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:10:03.939634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 606/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:10:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:10:03.962488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5933b57-ec21-4062-9c4e-61792da6051c None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:04.077053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5933b57-ec21-4062-9c4e-61792da6051c None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:04.116489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1813d6-11b8-4a0d-a909-b1b0fac9a416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:04.117452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1813d6-11b8-4a0d-a909-b1b0fac9a416 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:04.117761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1813d6-11b8-4a0d-a909-b1b0fac9a416 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:04.118115 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1813d6-11b8-4a0d-a909-b1b0fac9a416 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:04.131361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1813d6-11b8-4a0d-a909-b1b0fac9a416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-8b1813d6-11b8-4a0d-a909-b1b0fac9a416 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:10:04.140150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:04.141371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:04.142601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 616/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:10:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:10:04.148031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe59bff-40f8-4779-975e-c4e51107ac88 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:04.257002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe59bff-40f8-4779-975e-c4e51107ac88 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:04.285785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:04.299148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f041194-e53d-4ebf-972e-687f223c12c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:04.299816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f041194-e53d-4ebf-972e-687f223c12c1 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:04.300129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f041194-e53d-4ebf-972e-687f223c12c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:04.300413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f041194-e53d-4ebf-972e-687f223c12c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:04.301157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f041194-e53d-4ebf-972e-687f223c12c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:04.301464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f041194-e53d-4ebf-972e-687f223c12c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:04.303331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f041194-e53d-4ebf-972e-687f223c12c1 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:04.311306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 607/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:10:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:04.316994 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1813d6-11b8-4a0d-a909-b1b0fac9a416 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:04.363230 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:04.429827 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1813d6-11b8-4a0d-a909-b1b0fac9a416 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:04.468505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83626e-8c78-47a7-8353-2be5060160a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:04.469680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83626e-8c78-47a7-8353-2be5060160a5 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:04.470116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83626e-8c78-47a7-8353-2be5060160a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:04.470556 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83626e-8c78-47a7-8353-2be5060160a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:04.486444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83626e-8c78-47a7-8353-2be5060160a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:04.488702 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83626e-8c78-47a7-8353-2be5060160a5 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-ea83626e-8c78-47a7-8353-2be5060160a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:10:04.509395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:04.510394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:04.511320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:10:04.511320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:10:04.511320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:04.511320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:04.511320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 617/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:10:04.517415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f041194-e53d-4ebf-972e-687f223c12c1 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:04.634881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f041194-e53d-4ebf-972e-687f223c12c1 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:04.676543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f04f4-078d-4718-ad83-7f6518b312a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:04.677199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f04f4-078d-4718-ad83-7f6518b312a9 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:04.677997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f04f4-078d-4718-ad83-7f6518b312a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:04.678303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f04f4-078d-4718-ad83-7f6518b312a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:04.679015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f04f4-078d-4718-ad83-7f6518b312a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:04.679307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f04f4-078d-4718-ad83-7f6518b312a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:04.680697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f04f4-078d-4718-ad83-7f6518b312a9 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:04.692889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 608/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:04.699269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83626e-8c78-47a7-8353-2be5060160a5 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:04.806127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:04.813446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83626e-8c78-47a7-8353-2be5060160a5 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:04.848402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98976bdb-97d5-4636-8c5a-1f04b0492647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:04.849735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98976bdb-97d5-4636-8c5a-1f04b0492647 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:04.850104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98976bdb-97d5-4636-8c5a-1f04b0492647 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:04.850382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98976bdb-97d5-4636-8c5a-1f04b0492647 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:04.859121 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98976bdb-97d5-4636-8c5a-1f04b0492647 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:04.860666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98976bdb-97d5-4636-8c5a-1f04b0492647 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-98976bdb-97d5-4636-8c5a-1f04b0492647 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:10:04.876813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:04.878020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:04.879145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:04.879145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:04.879145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:10:04.879145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:10:04.879145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:10:04.879145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:10:04.879145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:10:04.879145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:10:04.879145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 07:10:04.879145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:04.879145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 618/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:04.885362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f04f4-078d-4718-ad83-7f6518b312a9 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:04.934963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:04.990474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f04f4-078d-4718-ad83-7f6518b312a9 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:05.028517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d1084-5a09-406e-9d85-e2f2a8f04999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:05.029177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d1084-5a09-406e-9d85-e2f2a8f04999 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:05.029469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d1084-5a09-406e-9d85-e2f2a8f04999 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:05.029748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d1084-5a09-406e-9d85-e2f2a8f04999 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:05.030590 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d1084-5a09-406e-9d85-e2f2a8f04999 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:05.030902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d1084-5a09-406e-9d85-e2f2a8f04999 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:05.032363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d1084-5a09-406e-9d85-e2f2a8f04999 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:05.051324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 609/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jun 17 07:10:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:05.057309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98976bdb-97d5-4636-8c5a-1f04b0492647 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:05.174637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98976bdb-97d5-4636-8c5a-1f04b0492647 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:05.213844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d71e4-4011-4819-899a-00f3d09b18ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:05.214665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d71e4-4011-4819-899a-00f3d09b18ca None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:05.214947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d71e4-4011-4819-899a-00f3d09b18ca None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:05.215219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d71e4-4011-4819-899a-00f3d09b18ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:05.226328 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d71e4-4011-4819-899a-00f3d09b18ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:05.227764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d71e4-4011-4819-899a-00f3d09b18ca None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-a06d71e4-4011-4819-899a-00f3d09b18ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:10:05.245308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:05.246428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:05.247472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:10:05.247472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:10:05.247472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:05.247472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:05.247472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 619/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:05.252265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d1084-5a09-406e-9d85-e2f2a8f04999 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:05.360547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d1084-5a09-406e-9d85-e2f2a8f04999 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:05.363439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:05.395622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc7fa24-ff83-4e35-92dc-1c2989a94f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:05.396405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc7fa24-ff83-4e35-92dc-1c2989a94f6e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:05.396692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc7fa24-ff83-4e35-92dc-1c2989a94f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:05.396935 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc7fa24-ff83-4e35-92dc-1c2989a94f6e None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:05.397655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc7fa24-ff83-4e35-92dc-1c2989a94f6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:05.397966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc7fa24-ff83-4e35-92dc-1c2989a94f6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:05.399340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc7fa24-ff83-4e35-92dc-1c2989a94f6e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:05.410809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 610/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:05.417021 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d71e4-4011-4819-899a-00f3d09b18ca None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:05.427132 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:05.523603 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d71e4-4011-4819-899a-00f3d09b18ca None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:05.558347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af1c6c-58b7-4440-9f72-8af30b5eedd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:05.559187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af1c6c-58b7-4440-9f72-8af30b5eedd6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:05.559507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af1c6c-58b7-4440-9f72-8af30b5eedd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:05.559784 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af1c6c-58b7-4440-9f72-8af30b5eedd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:05.569395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af1c6c-58b7-4440-9f72-8af30b5eedd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:05.570999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af1c6c-58b7-4440-9f72-8af30b5eedd6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-c0af1c6c-58b7-4440-9f72-8af30b5eedd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:10:05.587055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:05.588312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:05.589425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:10:05.589425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:10:05.589425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:10:05.589425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:10:05.589425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:10:05.589425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:10:05.589425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:10:05.589425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:05.589425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 620/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:10:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:05.595043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc7fa24-ff83-4e35-92dc-1c2989a94f6e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:05.700184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc7fa24-ff83-4e35-92dc-1c2989a94f6e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:05.735761 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a7405-60dc-4d38-ab76-341cdcc953b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:05.736446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a7405-60dc-4d38-ab76-341cdcc953b1 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:05.736732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a7405-60dc-4d38-ab76-341cdcc953b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:05.737005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a7405-60dc-4d38-ab76-341cdcc953b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:05.737775 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a7405-60dc-4d38-ab76-341cdcc953b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:05.738081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a7405-60dc-4d38-ab76-341cdcc953b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:05.739546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a7405-60dc-4d38-ab76-341cdcc953b1 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:05.751901 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 611/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:10:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:10:05.759020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af1c6c-58b7-4440-9f72-8af30b5eedd6 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:05.826352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:05.861576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af1c6c-58b7-4440-9f72-8af30b5eedd6 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:05.897232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41e67d-dcfe-4afb-9c57-7b0406889527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:05.898120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41e67d-dcfe-4afb-9c57-7b0406889527 None admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:05.898404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41e67d-dcfe-4afb-9c57-7b0406889527 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:05.898680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41e67d-dcfe-4afb-9c57-7b0406889527 None admin] PATH_INFO: `/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/b8cf02a52d354517b363c0aaf477550c/roles/2a15b4ff19204ac5916a0099ec8660c6` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:05.913482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41e67d-dcfe-4afb-9c57-7b0406889527 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4ec9b09302d4c918ef638d794468ba3, user_id=b8cf02a52d354517b363c0aaf477550c, role_id=2a15b4ff19204ac5916a0099ec8660c6)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:05.914922 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41e67d-dcfe-4afb-9c57-7b0406889527 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:05.929769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 621/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:10:05 2022] PUT /identity/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/b8cf02a52d354517b363c0aaf477550c/roles/2a15b4ff19204ac5916a0099ec8660c6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:10:08.171048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e758de2-88ee-46b9-8f16-b34e2efa202f None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:08.171622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e758de2-88ee-46b9-8f16-b34e2efa202f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:08.171622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e758de2-88ee-46b9-8f16-b34e2efa202f None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:08.172988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 612/1233] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:10:08.178248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3a8bf-4519-4505-9457-ee966ee89f2a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:08.178724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3a8bf-4519-4505-9457-ee966ee89f2a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:08.178838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3a8bf-4519-4505-9457-ee966ee89f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:08.197120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3df3a8bf-4519-4505-9457-ee966ee89f2a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:10:08.198387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3df3a8bf-4519-4505-9457-ee966ee89f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:10:08.269069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 622/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:10:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:10:08.294587 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e758de2-88ee-46b9-8f16-b34e2efa202f None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:08.320259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:08.410000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e758de2-88ee-46b9-8f16-b34e2efa202f None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:08.449065 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0bade-5457-49ed-9aa0-d0fa493bd404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:08.449902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0bade-5457-49ed-9aa0-d0fa493bd404 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:08.450191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0bade-5457-49ed-9aa0-d0fa493bd404 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:08.450506 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0bade-5457-49ed-9aa0-d0fa493bd404 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:08.460809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0bade-5457-49ed-9aa0-d0fa493bd404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-2bb0bade-5457-49ed-9aa0-d0fa493bd404 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:10:08.469152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:08.470405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:08.471360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 613/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jun 17 07:10:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:08.477471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3a8bf-4519-4505-9457-ee966ee89f2a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:08.525544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:08.583253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3a8bf-4519-4505-9457-ee966ee89f2a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:08.622054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f62b4e-9f44-4942-98b6-ef33735129b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:08.622835 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f62b4e-9f44-4942-98b6-ef33735129b7 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:08.623151 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f62b4e-9f44-4942-98b6-ef33735129b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:08.623435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f62b4e-9f44-4942-98b6-ef33735129b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:08.624365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f62b4e-9f44-4942-98b6-ef33735129b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:08.624804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f62b4e-9f44-4942-98b6-ef33735129b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:08.626432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f62b4e-9f44-4942-98b6-ef33735129b7 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:08.636878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 623/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jun 17 07:10:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:08.643578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0bade-5457-49ed-9aa0-d0fa493bd404 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:08.737404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:08.754092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0bade-5457-49ed-9aa0-d0fa493bd404 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:08.792815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dd649-ab6a-40c6-b06d-61b2edc55b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:08.793715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dd649-ab6a-40c6-b06d-61b2edc55b71 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:08.794056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dd649-ab6a-40c6-b06d-61b2edc55b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:08.794439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dd649-ab6a-40c6-b06d-61b2edc55b71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:08.806318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dd649-ab6a-40c6-b06d-61b2edc55b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:08.808072 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dd649-ab6a-40c6-b06d-61b2edc55b71 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-097dd649-ab6a-40c6-b06d-61b2edc55b71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:10:08.826507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:08.827402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:08.828557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:10:08.828557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:10:08.828557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:08.828557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:08.828557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 614/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:08.834955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f62b4e-9f44-4942-98b6-ef33735129b7 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:08.945166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f62b4e-9f44-4942-98b6-ef33735129b7 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:08.955475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:08.984676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfeb9f7-bbe5-45f4-84d6-375b3bec85c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:08.985466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeb9f7-bbe5-45f4-84d6-375b3bec85c7 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:08.985787 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeb9f7-bbe5-45f4-84d6-375b3bec85c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:08.986071 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeb9f7-bbe5-45f4-84d6-375b3bec85c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:08.986952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfeb9f7-bbe5-45f4-84d6-375b3bec85c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:08.987580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfeb9f7-bbe5-45f4-84d6-375b3bec85c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:08.989487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfeb9f7-bbe5-45f4-84d6-375b3bec85c7 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:09.004520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 624/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:09.011629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dd649-ab6a-40c6-b06d-61b2edc55b71 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:09.120139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dd649-ab6a-40c6-b06d-61b2edc55b71 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:09.156509 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f4ed-1129-44f0-9099-5e89168ae422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:09.157343 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f4ed-1129-44f0-9099-5e89168ae422 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:09.157650 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f4ed-1129-44f0-9099-5e89168ae422 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:09.158062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f4ed-1129-44f0-9099-5e89168ae422 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:09.161145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:09.168617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f4ed-1129-44f0-9099-5e89168ae422 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:09.170948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f4ed-1129-44f0-9099-5e89168ae422 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-1609f4ed-1129-44f0-9099-5e89168ae422 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:10:09.187897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:09.190560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:09.191606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:09.191606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:09.191606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:10:09.191606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:10:09.191606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:10:09.191606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:10:09.191606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:10:09.191606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:10:09.191606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 07:10:09.191606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:09.191606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 615/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 07:10:09.195676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfeb9f7-bbe5-45f4-84d6-375b3bec85c7 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:09.320296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfeb9f7-bbe5-45f4-84d6-375b3bec85c7 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:09.356369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7ed61-5621-409d-89cc-76898816108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:09.357057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7ed61-5621-409d-89cc-76898816108c None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:09.357347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7ed61-5621-409d-89cc-76898816108c None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:09.357641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7ed61-5621-409d-89cc-76898816108c None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:09.358473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7ed61-5621-409d-89cc-76898816108c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:09.358780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7ed61-5621-409d-89cc-76898816108c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:09.360245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7ed61-5621-409d-89cc-76898816108c None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:09.377949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 625/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jun 17 07:10:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:09.383622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f4ed-1129-44f0-9099-5e89168ae422 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:09.497042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f4ed-1129-44f0-9099-5e89168ae422 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:09.534438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8bb6d-a106-4004-86f5-43e1ef6b20c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:09.535106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8bb6d-a106-4004-86f5-43e1ef6b20c3 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:09.535399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8bb6d-a106-4004-86f5-43e1ef6b20c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:09.535677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8bb6d-a106-4004-86f5-43e1ef6b20c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:09.607726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:09.615173 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8bb6d-a106-4004-86f5-43e1ef6b20c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:09.616661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8bb6d-a106-4004-86f5-43e1ef6b20c3 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:09.718123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 616/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:10:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:10:09.724165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7ed61-5621-409d-89cc-76898816108c None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:09.759999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:09.840138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7ed61-5621-409d-89cc-76898816108c None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:09.875573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a45903-5228-44fa-be71-1540dc3e7565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:09.876350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a45903-5228-44fa-be71-1540dc3e7565 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:09.876636 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a45903-5228-44fa-be71-1540dc3e7565 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:09.876912 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a45903-5228-44fa-be71-1540dc3e7565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:09.888115 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a45903-5228-44fa-be71-1540dc3e7565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:09.889672 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a45903-5228-44fa-be71-1540dc3e7565 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-68a45903-5228-44fa-be71-1540dc3e7565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:10:09.908920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:09.910016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:09.911253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:10:09.911253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:10:09.911253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:09.911253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:09.911527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 626/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:09.919597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8bb6d-a106-4004-86f5-43e1ef6b20c3 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:10.024454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:10.030870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8bb6d-a106-4004-86f5-43e1ef6b20c3 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:10.068257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1b549-c78c-4c64-ab50-a0726bd16d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:10.069118 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1b549-c78c-4c64-ab50-a0726bd16d5e None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:10.069440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1b549-c78c-4c64-ab50-a0726bd16d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:10.069837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1b549-c78c-4c64-ab50-a0726bd16d5e None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:10.070599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1b549-c78c-4c64-ab50-a0726bd16d5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:10.070966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1b549-c78c-4c64-ab50-a0726bd16d5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:10.072802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1b549-c78c-4c64-ab50-a0726bd16d5e None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:10.087618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 617/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:10.093442 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a45903-5228-44fa-be71-1540dc3e7565 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:10.186469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:10.223710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a45903-5228-44fa-be71-1540dc3e7565 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:10.280893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9d67c-7ecb-4344-8c5d-a81e0b251070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:10.282028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9d67c-7ecb-4344-8c5d-a81e0b251070 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:10.282511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9d67c-7ecb-4344-8c5d-a81e0b251070 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:10.282991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9d67c-7ecb-4344-8c5d-a81e0b251070 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:10.292369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9d67c-7ecb-4344-8c5d-a81e0b251070 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:10.294070 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9d67c-7ecb-4344-8c5d-a81e0b251070 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-d3e9d67c-7ecb-4344-8c5d-a81e0b251070 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:10:10.313577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:10.314585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:10.315494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:10:10.315494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:10:10.315494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:10:10.315494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:10:10.315494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:10:10.315494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:10:10.315494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:10:10.315494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:10.315494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 627/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:10:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:10.322097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1b549-c78c-4c64-ab50-a0726bd16d5e None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:10.443945 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1b549-c78c-4c64-ab50-a0726bd16d5e None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:10.494427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342c653-ba9f-4310-b19b-fddf1faaada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:10.495104 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342c653-ba9f-4310-b19b-fddf1faaada6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:10.495640 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342c653-ba9f-4310-b19b-fddf1faaada6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:10.495794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342c653-ba9f-4310-b19b-fddf1faaada6 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:10.496790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342c653-ba9f-4310-b19b-fddf1faaada6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:10.497156 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342c653-ba9f-4310-b19b-fddf1faaada6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:10.499262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342c653-ba9f-4310-b19b-fddf1faaada6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:10.517399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 618/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:10:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:10.523853 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9d67c-7ecb-4344-8c5d-a81e0b251070 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:10.629878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9d67c-7ecb-4344-8c5d-a81e0b251070 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:10.663626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081b61d-981b-41ae-a419-6f255e1dc1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:10.664301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081b61d-981b-41ae-a419-6f255e1dc1fa None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:10.664588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081b61d-981b-41ae-a419-6f255e1dc1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:10.664893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081b61d-981b-41ae-a419-6f255e1dc1fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:10.665690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081b61d-981b-41ae-a419-6f255e1dc1fa None admin] RBAC: Adding query filter params (role.id=2a15b4ff19204ac5916a0099ec8660c6, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=b8cf02a52d354517b363c0aaf477550c) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:10.666009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081b61d-981b-41ae-a419-6f255e1dc1fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:10.667430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081b61d-981b-41ae-a419-6f255e1dc1fa None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:10.673548 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 628/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:10:10 2022] GET /identity/v3/role_assignments?user.id=b8cf02a52d354517b363c0aaf477550c&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=2a15b4ff19204ac5916a0099ec8660c6 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:12.949065 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02374f29-009b-44dc-8ee9-cff4e678b0b0 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:12.949690 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02374f29-009b-44dc-8ee9-cff4e678b0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:12.949690 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02374f29-009b-44dc-8ee9-cff4e678b0b0 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:12.951381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 619/1247] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:10:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:10:12.957544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9972eb-be14-4cbb-bb6b-a4ad8c040988 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:12.958017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9972eb-be14-4cbb-bb6b-a4ad8c040988 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:12.958116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9972eb-be14-4cbb-bb6b-a4ad8c040988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:12.964137 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:12.978886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ed9972eb-be14-4cbb-bb6b-a4ad8c040988 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:10:12.980258 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ed9972eb-be14-4cbb-bb6b-a4ad8c040988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:10:13.058478 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 629/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:10:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:10:13.081802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02374f29-009b-44dc-8ee9-cff4e678b0b0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:13.103396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:13.194735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02374f29-009b-44dc-8ee9-cff4e678b0b0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:13.244968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02884c-a9e8-4134-8252-711abe6c7260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:13.245847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02884c-a9e8-4134-8252-711abe6c7260 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:13.246248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02884c-a9e8-4134-8252-711abe6c7260 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:13.246696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02884c-a9e8-4134-8252-711abe6c7260 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:13.260927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02884c-a9e8-4134-8252-711abe6c7260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-0b02884c-a9e8-4134-8252-711abe6c7260 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:10:13.270147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:13.272671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:13.274585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 620/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:10:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:13.278862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9972eb-be14-4cbb-bb6b-a4ad8c040988 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:13.393184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9972eb-be14-4cbb-bb6b-a4ad8c040988 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:13.427471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:13.434431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d51bcf-40cc-4f9e-b4af-a21f3ad8ace6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:13.435147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d51bcf-40cc-4f9e-b4af-a21f3ad8ace6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:13.435398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d51bcf-40cc-4f9e-b4af-a21f3ad8ace6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:13.435678 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d51bcf-40cc-4f9e-b4af-a21f3ad8ace6 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:13.436451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d51bcf-40cc-4f9e-b4af-a21f3ad8ace6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:13.436762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d51bcf-40cc-4f9e-b4af-a21f3ad8ace6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:13.438137 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d51bcf-40cc-4f9e-b4af-a21f3ad8ace6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:13.453403 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 630/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:10:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:10:13.454593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02884c-a9e8-4134-8252-711abe6c7260 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:13.540496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:13.566154 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02884c-a9e8-4134-8252-711abe6c7260 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:13.601464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baed92-5624-4866-8844-cb1437ebdddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:13.602297 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baed92-5624-4866-8844-cb1437ebdddb None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:13.602584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baed92-5624-4866-8844-cb1437ebdddb None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:13.602864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baed92-5624-4866-8844-cb1437ebdddb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:13.615846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4baed92-5624-4866-8844-cb1437ebdddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:13.617777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4baed92-5624-4866-8844-cb1437ebdddb None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-d4baed92-5624-4866-8844-cb1437ebdddb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:10:13.638974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:13.640096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:13.641143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:10:13.641143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:10:13.641143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:13.641143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:13.642681 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 621/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:13.649324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d51bcf-40cc-4f9e-b4af-a21f3ad8ace6 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:13.762979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d51bcf-40cc-4f9e-b4af-a21f3ad8ace6 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:13.799010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901328d-bd8a-477c-b265-d23d39b38494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:13.799626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901328d-bd8a-477c-b265-d23d39b38494 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:13.799908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901328d-bd8a-477c-b265-d23d39b38494 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:13.800219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901328d-bd8a-477c-b265-d23d39b38494 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:13.801051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901328d-bd8a-477c-b265-d23d39b38494 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:13.801365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901328d-bd8a-477c-b265-d23d39b38494 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:13.803261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901328d-bd8a-477c-b265-d23d39b38494 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:13.815538 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 631/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:13.823230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baed92-5624-4866-8844-cb1437ebdddb None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:13.926898 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baed92-5624-4866-8844-cb1437ebdddb None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:13.954934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:13.966971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be66ab2-71ab-499b-a7c3-e78e2aa77753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:13.968092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be66ab2-71ab-499b-a7c3-e78e2aa77753 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:13.968382 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be66ab2-71ab-499b-a7c3-e78e2aa77753 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:13.968662 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be66ab2-71ab-499b-a7c3-e78e2aa77753 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:13.976570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be66ab2-71ab-499b-a7c3-e78e2aa77753 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:13.978007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be66ab2-71ab-499b-a7c3-e78e2aa77753 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-6be66ab2-71ab-499b-a7c3-e78e2aa77753 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:10:13.992840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:13.994078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:13.995226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:13.995226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:13.995226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:10:13.995226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:10:13.995226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:10:13.995226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:10:13.995226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:10:13.995226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:10:13.995226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 07:10:13.995226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:13.995226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 622/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:13.999901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901328d-bd8a-477c-b265-d23d39b38494 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:14.057367 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:14.110083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901328d-bd8a-477c-b265-d23d39b38494 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:14.147727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193943df-8e0f-4a16-aebd-87b2d4109950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:14.148509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193943df-8e0f-4a16-aebd-87b2d4109950 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:14.148799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193943df-8e0f-4a16-aebd-87b2d4109950 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:14.149222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193943df-8e0f-4a16-aebd-87b2d4109950 None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:14.150047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193943df-8e0f-4a16-aebd-87b2d4109950 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:14.150370 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193943df-8e0f-4a16-aebd-87b2d4109950 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:14.152039 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193943df-8e0f-4a16-aebd-87b2d4109950 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:14.171443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 632/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jun 17 07:10:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:10:14.177280 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be66ab2-71ab-499b-a7c3-e78e2aa77753 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:14.296449 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be66ab2-71ab-499b-a7c3-e78e2aa77753 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:14.329951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fad5c6-2fcb-4897-8974-76be41052dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:14.330530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fad5c6-2fcb-4897-8974-76be41052dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:14.330818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fad5c6-2fcb-4897-8974-76be41052dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:14.331096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fad5c6-2fcb-4897-8974-76be41052dc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:14.391783 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:14.406591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fad5c6-2fcb-4897-8974-76be41052dc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:14.408034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fad5c6-2fcb-4897-8974-76be41052dc4 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:14.517301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 623/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:10:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:10:14.518596 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193943df-8e0f-4a16-aebd-87b2d4109950 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:14.622813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193943df-8e0f-4a16-aebd-87b2d4109950 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:14.638378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:14.665642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba0c4f-5a1d-44fd-aad9-98439b7cc72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:14.666511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba0c4f-5a1d-44fd-aad9-98439b7cc72b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:14.666794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba0c4f-5a1d-44fd-aad9-98439b7cc72b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:14.667065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba0c4f-5a1d-44fd-aad9-98439b7cc72b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:14.680920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba0c4f-5a1d-44fd-aad9-98439b7cc72b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:14.682576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba0c4f-5a1d-44fd-aad9-98439b7cc72b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-5aba0c4f-5a1d-44fd-aad9-98439b7cc72b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:10:14.700440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:14.702309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:14.704111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:10:14.704111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:10:14.704111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:14.704111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:14.704111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 633/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:14.707504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fad5c6-2fcb-4897-8974-76be41052dc4 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:14.809216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:14.823393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fad5c6-2fcb-4897-8974-76be41052dc4 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:14.860876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc39855-6396-4174-8029-8a5c63eb6f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:14.861561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc39855-6396-4174-8029-8a5c63eb6f53 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:14.861852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc39855-6396-4174-8029-8a5c63eb6f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:14.862134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc39855-6396-4174-8029-8a5c63eb6f53 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:14.862878 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc39855-6396-4174-8029-8a5c63eb6f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:14.863190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc39855-6396-4174-8029-8a5c63eb6f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:14.865164 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc39855-6396-4174-8029-8a5c63eb6f53 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:14.883401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 624/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:10:14.884743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba0c4f-5a1d-44fd-aad9-98439b7cc72b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:14.988744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba0c4f-5a1d-44fd-aad9-98439b7cc72b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:15.024249 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271878f-9ab7-4a02-9a1e-d163ab15a220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:15.025058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271878f-9ab7-4a02-9a1e-d163ab15a220 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:15.025340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271878f-9ab7-4a02-9a1e-d163ab15a220 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:15.025614 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271878f-9ab7-4a02-9a1e-d163ab15a220 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:15.033108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271878f-9ab7-4a02-9a1e-d163ab15a220 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:15.034576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271878f-9ab7-4a02-9a1e-d163ab15a220 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-d271878f-9ab7-4a02-9a1e-d163ab15a220 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:10:15.049116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:15.050156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:15.051165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:10:15.051165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:10:15.051165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:10:15.051165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:10:15.051165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:10:15.051165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:10:15.051165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:10:15.051165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:15.051165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 634/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:10:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:15.056456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc39855-6396-4174-8029-8a5c63eb6f53 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:15.162049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc39855-6396-4174-8029-8a5c63eb6f53 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:15.206386 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bde693-df06-42be-a2a4-24259c418ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:15.207344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde693-df06-42be-a2a4-24259c418ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:15.207749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde693-df06-42be-a2a4-24259c418ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:15.208303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde693-df06-42be-a2a4-24259c418ff6 None admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:15.209802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bde693-df06-42be-a2a4-24259c418ff6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:15.210130 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bde693-df06-42be-a2a4-24259c418ff6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:15.211605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bde693-df06-42be-a2a4-24259c418ff6 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:15.225504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 625/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:10:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:15.231997 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271878f-9ab7-4a02-9a1e-d163ab15a220 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:15.263144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:15.335785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271878f-9ab7-4a02-9a1e-d163ab15a220 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:15.372726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738621d-2fa6-4486-8f14-8f250b3c496e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:15.373383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738621d-2fa6-4486-8f14-8f250b3c496e None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:15.373699 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738621d-2fa6-4486-8f14-8f250b3c496e None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:15.373980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738621d-2fa6-4486-8f14-8f250b3c496e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:15.374819 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738621d-2fa6-4486-8f14-8f250b3c496e None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=b8cf02a52d354517b363c0aaf477550c) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:15.375141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738621d-2fa6-4486-8f14-8f250b3c496e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:15.376631 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738621d-2fa6-4486-8f14-8f250b3c496e None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:15.384122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 635/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:10:15 2022] GET /identity/v3/role_assignments?user.id=b8cf02a52d354517b363c0aaf477550c&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:17.465152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10830c-59a2-4254-9ca5-4168d43fa958 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:17.465695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10830c-59a2-4254-9ca5-4168d43fa958 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:17.465695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10830c-59a2-4254-9ca5-4168d43fa958 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:17.467066 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 626/1261] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:10:17.472056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17363742-a5d4-44a3-8f4e-d0edc61c662f None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:17.472532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17363742-a5d4-44a3-8f4e-d0edc61c662f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:17.472651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17363742-a5d4-44a3-8f4e-d0edc61c662f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:17.488840 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-17363742-a5d4-44a3-8f4e-d0edc61c662f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:10:17.490530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-17363742-a5d4-44a3-8f4e-d0edc61c662f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:10:17.554438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 636/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:10:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:10:17.576377 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10830c-59a2-4254-9ca5-4168d43fa958 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:17.586767 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:17.673648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10830c-59a2-4254-9ca5-4168d43fa958 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:17.705168 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42b581-13d6-4100-b1f8-d80a19266c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:17.705945 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42b581-13d6-4100-b1f8-d80a19266c4f None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:17.706231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42b581-13d6-4100-b1f8-d80a19266c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:17.706510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42b581-13d6-4100-b1f8-d80a19266c4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:17.716744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42b581-13d6-4100-b1f8-d80a19266c4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-6c42b581-13d6-4100-b1f8-d80a19266c4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:10:17.724447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:17.725581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:17.726458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 627/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:10:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:17.731484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17363742-a5d4-44a3-8f4e-d0edc61c662f None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:17.749633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:17.827407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17363742-a5d4-44a3-8f4e-d0edc61c662f None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:17.859178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3ad97-6fc6-415e-93db-bf49a47c07e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:17.859853 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3ad97-6fc6-415e-93db-bf49a47c07e4 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:17.860166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3ad97-6fc6-415e-93db-bf49a47c07e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:17.860448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3ad97-6fc6-415e-93db-bf49a47c07e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:17.861181 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3ad97-6fc6-415e-93db-bf49a47c07e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:17.861490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3ad97-6fc6-415e-93db-bf49a47c07e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:17.862844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3ad97-6fc6-415e-93db-bf49a47c07e4 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:17.871080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 637/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:10:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:17.876476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42b581-13d6-4100-b1f8-d80a19266c4f None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:17.941500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:17.973506 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42b581-13d6-4100-b1f8-d80a19266c4f None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:18.006109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c7e7f-0a56-40e1-af84-dd129c8a168c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:18.006886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c7e7f-0a56-40e1-af84-dd129c8a168c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:18.007174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c7e7f-0a56-40e1-af84-dd129c8a168c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:18.007455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c7e7f-0a56-40e1-af84-dd129c8a168c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:18.017557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c7e7f-0a56-40e1-af84-dd129c8a168c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:18.018998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c7e7f-0a56-40e1-af84-dd129c8a168c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-675c7e7f-0a56-40e1-af84-dd129c8a168c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:10:18.035782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:18.037181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:18.038124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:10:18.038124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:10:18.038124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:18.038124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:18.038124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 628/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:18.042980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3ad97-6fc6-415e-93db-bf49a47c07e4 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:18.120973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:18.139270 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3ad97-6fc6-415e-93db-bf49a47c07e4 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:18.171395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589555a-9704-49e1-8f3b-664a1e48dd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:18.172036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589555a-9704-49e1-8f3b-664a1e48dd74 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:18.172351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589555a-9704-49e1-8f3b-664a1e48dd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:18.172627 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589555a-9704-49e1-8f3b-664a1e48dd74 None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:18.173337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589555a-9704-49e1-8f3b-664a1e48dd74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:18.173644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589555a-9704-49e1-8f3b-664a1e48dd74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:18.174987 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589555a-9704-49e1-8f3b-664a1e48dd74 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:18.191920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 638/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:18.193369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c7e7f-0a56-40e1-af84-dd129c8a168c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:18.292743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c7e7f-0a56-40e1-af84-dd129c8a168c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:18.311192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:18.324605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33475073-7ef2-4b08-b898-b2ed4737b91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:18.325399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33475073-7ef2-4b08-b898-b2ed4737b91c None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:18.325680 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33475073-7ef2-4b08-b898-b2ed4737b91c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:18.325953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33475073-7ef2-4b08-b898-b2ed4737b91c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:18.334285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33475073-7ef2-4b08-b898-b2ed4737b91c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:18.335720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33475073-7ef2-4b08-b898-b2ed4737b91c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-33475073-7ef2-4b08-b898-b2ed4737b91c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:10:18.349425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:18.350351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:18.351327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:18.351327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:18.351327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:10:18.351327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:10:18.351327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:10:18.351327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:10:18.351327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:10:18.351327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:10:18.351327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 07:10:18.351327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:18.351327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 629/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:18.356369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589555a-9704-49e1-8f3b-664a1e48dd74 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:18.452982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589555a-9704-49e1-8f3b-664a1e48dd74 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:18.485685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188939b-c177-4535-8d6a-4abe86187a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:18.486338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188939b-c177-4535-8d6a-4abe86187a4a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:18.486626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188939b-c177-4535-8d6a-4abe86187a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:18.486908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188939b-c177-4535-8d6a-4abe86187a4a None admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:18.487734 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188939b-c177-4535-8d6a-4abe86187a4a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:18.488048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188939b-c177-4535-8d6a-4abe86187a4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:18.489496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188939b-c177-4535-8d6a-4abe86187a4a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:18.498334 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:18.507141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 639/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jun 17 07:10:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:18.512745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33475073-7ef2-4b08-b898-b2ed4737b91c None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:18.607711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33475073-7ef2-4b08-b898-b2ed4737b91c None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:18.639938 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614468d-e15c-4cfd-8a8a-e9f49d1bf077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:18.640735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614468d-e15c-4cfd-8a8a-e9f49d1bf077 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:18.641020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614468d-e15c-4cfd-8a8a-e9f49d1bf077 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:18.641294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614468d-e15c-4cfd-8a8a-e9f49d1bf077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:18.651355 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614468d-e15c-4cfd-8a8a-e9f49d1bf077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:18.653071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614468d-e15c-4cfd-8a8a-e9f49d1bf077 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-f614468d-e15c-4cfd-8a8a-e9f49d1bf077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:10:18.669958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:18.670983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:18.672049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:10:18.672049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:10:18.672049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:18.672049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:18.672049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 630/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:18.676596 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188939b-c177-4535-8d6a-4abe86187a4a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:18.770971 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188939b-c177-4535-8d6a-4abe86187a4a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:18.803807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ac166-0a18-4545-ad60-59c077d4e92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:18.804473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ac166-0a18-4545-ad60-59c077d4e92a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:18.804756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ac166-0a18-4545-ad60-59c077d4e92a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:18.805030 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ac166-0a18-4545-ad60-59c077d4e92a None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:18.805736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ac166-0a18-4545-ad60-59c077d4e92a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:18.806042 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ac166-0a18-4545-ad60-59c077d4e92a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:18.807462 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ac166-0a18-4545-ad60-59c077d4e92a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:18.818145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 640/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:18.823465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614468d-e15c-4cfd-8a8a-e9f49d1bf077 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:18.854171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:18.919954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614468d-e15c-4cfd-8a8a-e9f49d1bf077 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:18.951852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463c871-7a64-4f9d-a54b-e577a754b3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:18.952658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463c871-7a64-4f9d-a54b-e577a754b3ff None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:18.952948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463c871-7a64-4f9d-a54b-e577a754b3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:18.953227 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463c871-7a64-4f9d-a54b-e577a754b3ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:18.960179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463c871-7a64-4f9d-a54b-e577a754b3ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:18.961608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463c871-7a64-4f9d-a54b-e577a754b3ff None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-e463c871-7a64-4f9d-a54b-e577a754b3ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:10:18.975345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:18.976440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:18.977561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:10:18.977561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:10:18.977561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:10:18.977561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:10:18.977561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:10:18.977561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:10:18.977561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:10:18.977561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:18.977561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 631/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:10:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:18.982189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ac166-0a18-4545-ad60-59c077d4e92a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:19.038773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:19.079415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ac166-0a18-4545-ad60-59c077d4e92a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:19.111153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dbd69-b788-460b-b735-44ccab7684a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:19.111795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dbd69-b788-460b-b735-44ccab7684a6 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:19.112101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dbd69-b788-460b-b735-44ccab7684a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:19.112388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dbd69-b788-460b-b735-44ccab7684a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:19.113167 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dbd69-b788-460b-b735-44ccab7684a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:19.113479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dbd69-b788-460b-b735-44ccab7684a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:19.115400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dbd69-b788-460b-b735-44ccab7684a6 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:19.126955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 641/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:10:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:10:19.133250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463c871-7a64-4f9d-a54b-e577a754b3ff None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:19.225555 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:19.234445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463c871-7a64-4f9d-a54b-e577a754b3ff None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:19.267614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d4a29-5cd8-41c7-8da5-882d413bc57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:19.268521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d4a29-5cd8-41c7-8da5-882d413bc57c None admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:19.268811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d4a29-5cd8-41c7-8da5-882d413bc57c None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:19.270053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d4a29-5cd8-41c7-8da5-882d413bc57c None admin] PATH_INFO: `/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/b8cf02a52d354517b363c0aaf477550c/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:19.272033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d4a29-5cd8-41c7-8da5-882d413bc57c None admin] RBAC: Authorizing `identity:create_grant(project_id=a4ec9b09302d4c918ef638d794468ba3, user_id=b8cf02a52d354517b363c0aaf477550c, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:19.273490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d4a29-5cd8-41c7-8da5-882d413bc57c None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:19.285645 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 632/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jun 17 07:10:19 2022] PUT /identity/v3/projects/a4ec9b09302d4c918ef638d794468ba3/users/b8cf02a52d354517b363c0aaf477550c/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:10:21.338991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3aa3c-b926-4272-9acb-c183f7a945cc None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:21.339521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3aa3c-b926-4272-9acb-c183f7a945cc None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:21.339521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3aa3c-b926-4272-9acb-c183f7a945cc None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:21.340926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 642/1274] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:10:21.346017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95264632-9c42-4788-a976-e3bb22e722d0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:21.346406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95264632-9c42-4788-a976-e3bb22e722d0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:21.346585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95264632-9c42-4788-a976-e3bb22e722d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:21.362829 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-95264632-9c42-4788-a976-e3bb22e722d0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:10:21.364015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-95264632-9c42-4788-a976-e3bb22e722d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:10:21.429569 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 633/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:10:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:10:21.454769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3aa3c-b926-4272-9acb-c183f7a945cc None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:21.551236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3aa3c-b926-4272-9acb-c183f7a945cc None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:21.568808 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:21.585177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406a64a-229c-4952-b8ab-7938e9db1deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:21.585951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406a64a-229c-4952-b8ab-7938e9db1deb None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:21.586235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406a64a-229c-4952-b8ab-7938e9db1deb None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:21.586508 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406a64a-229c-4952-b8ab-7938e9db1deb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:21.596276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406a64a-229c-4952-b8ab-7938e9db1deb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-e406a64a-229c-4952-b8ab-7938e9db1deb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 07:10:21.603741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:21.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:21.606069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 643/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jun 17 07:10:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:21.610893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95264632-9c42-4788-a976-e3bb22e722d0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:21.689741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:21.707821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95264632-9c42-4788-a976-e3bb22e722d0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:21.741213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cbe30-9991-4600-b135-e2c50cc5c6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:21.741864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cbe30-9991-4600-b135-e2c50cc5c6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:21.742145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cbe30-9991-4600-b135-e2c50cc5c6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:21.742416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cbe30-9991-4600-b135-e2c50cc5c6e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:21.743139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cbe30-9991-4600-b135-e2c50cc5c6e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:21.743447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cbe30-9991-4600-b135-e2c50cc5c6e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:21.744830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cbe30-9991-4600-b135-e2c50cc5c6e8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:21.752465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 634/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jun 17 07:10:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:21.758194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406a64a-229c-4952-b8ab-7938e9db1deb None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:21.853744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406a64a-229c-4952-b8ab-7938e9db1deb None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:21.886168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed2f58-1915-431d-9c4b-64f580069a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:21.886928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2f58-1915-431d-9c4b-64f580069a8b None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:21.887210 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2f58-1915-431d-9c4b-64f580069a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:21.887485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2f58-1915-431d-9c4b-64f580069a8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:21.897533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed2f58-1915-431d-9c4b-64f580069a8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:21.898962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed2f58-1915-431d-9c4b-64f580069a8b None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-e7ed2f58-1915-431d-9c4b-64f580069a8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:10:21.916118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:21.917145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:21.918172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:10:21.918172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:10:21.918172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:21.918172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:21.918172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 644/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:21.922816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cbe30-9991-4600-b135-e2c50cc5c6e8 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:22.018465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cbe30-9991-4600-b135-e2c50cc5c6e8 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:22.050980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76045885-5ad3-44aa-9da0-8444a77b9ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:22.051628 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76045885-5ad3-44aa-9da0-8444a77b9ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:22.051915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76045885-5ad3-44aa-9da0-8444a77b9ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:22.052778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76045885-5ad3-44aa-9da0-8444a77b9ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:22.053492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76045885-5ad3-44aa-9da0-8444a77b9ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:22.053800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76045885-5ad3-44aa-9da0-8444a77b9ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:22.055162 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76045885-5ad3-44aa-9da0-8444a77b9ea4 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:22.066330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 635/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:22.072395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed2f58-1915-431d-9c4b-64f580069a8b None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:22.102549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:22.170065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed2f58-1915-431d-9c4b-64f580069a8b None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:22.204192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d523b3c-f140-4cbf-a5b8-8f1d6b875459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:22.205147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d523b3c-f140-4cbf-a5b8-8f1d6b875459 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:22.205464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d523b3c-f140-4cbf-a5b8-8f1d6b875459 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:22.205774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d523b3c-f140-4cbf-a5b8-8f1d6b875459 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:22.216640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d523b3c-f140-4cbf-a5b8-8f1d6b875459 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:22.218487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d523b3c-f140-4cbf-a5b8-8f1d6b875459 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-6d523b3c-f140-4cbf-a5b8-8f1d6b875459 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 07:10:22.237669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:22.238860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:22.240059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:22.240059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:22.240059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 07:10:22.240059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 07:10:22.240059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 07:10:22.240059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 07:10:22.240059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 07:10:22.240059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 07:10:22.240059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 07:10:22.240059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:22.240059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 645/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:22.245459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76045885-5ad3-44aa-9da0-8444a77b9ea4 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:22.250476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:22.345210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76045885-5ad3-44aa-9da0-8444a77b9ea4 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:22.377952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295c639-3fc1-4ca6-b690-6e059980dca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:22.378614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295c639-3fc1-4ca6-b690-6e059980dca8 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:22.378901 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295c639-3fc1-4ca6-b690-6e059980dca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:22.379182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295c639-3fc1-4ca6-b690-6e059980dca8 None admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:22.380010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295c639-3fc1-4ca6-b690-6e059980dca8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:22.380869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295c639-3fc1-4ca6-b690-6e059980dca8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:22.382294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295c639-3fc1-4ca6-b690-6e059980dca8 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:22.399694 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 636/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jun 17 07:10:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:22.405706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d523b3c-f140-4cbf-a5b8-8f1d6b875459 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:22.492417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:22.501732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d523b3c-f140-4cbf-a5b8-8f1d6b875459 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:22.534279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af9073-0374-4956-942f-1e38296d874a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:22.535160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af9073-0374-4956-942f-1e38296d874a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:22.535553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af9073-0374-4956-942f-1e38296d874a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:22.535941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af9073-0374-4956-942f-1e38296d874a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:22.605263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af9073-0374-4956-942f-1e38296d874a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:22.606692 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af9073-0374-4956-942f-1e38296d874a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:22.689931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:22.700388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 646/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jun 17 07:10:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 07:10:22.706135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295c639-3fc1-4ca6-b690-6e059980dca8 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:22.765448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:22.803142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295c639-3fc1-4ca6-b690-6e059980dca8 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:22.835305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6508e-88a2-4b43-9406-32b443f08874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:22.836100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6508e-88a2-4b43-9406-32b443f08874 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:22.836387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6508e-88a2-4b43-9406-32b443f08874 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:22.836663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6508e-88a2-4b43-9406-32b443f08874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:22.846219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6508e-88a2-4b43-9406-32b443f08874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:22.847650 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6508e-88a2-4b43-9406-32b443f08874 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-65c6508e-88a2-4b43-9406-32b443f08874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 07:10:22.866836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:22.867904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:22.868892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 07:10:22.868892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 07:10:22.868892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 07:10:22.868892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:22.868892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 637/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:22.873505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af9073-0374-4956-942f-1e38296d874a None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:22.967386 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af9073-0374-4956-942f-1e38296d874a None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:22.999627 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e324f1d-0615-4b05-b3a4-dc58d7bde99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:23.000297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e324f1d-0615-4b05-b3a4-dc58d7bde99d None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:23.000581 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e324f1d-0615-4b05-b3a4-dc58d7bde99d None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:23.000858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e324f1d-0615-4b05-b3a4-dc58d7bde99d None admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:23.001564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e324f1d-0615-4b05-b3a4-dc58d7bde99d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:23.001871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e324f1d-0615-4b05-b3a4-dc58d7bde99d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:23.003222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e324f1d-0615-4b05-b3a4-dc58d7bde99d None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:23.013904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 647/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jun 17 07:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:23.019436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6508e-88a2-4b43-9406-32b443f08874 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:23.115214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6508e-88a2-4b43-9406-32b443f08874 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:23.128473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:23.149078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b8c92-ff74-4239-b298-2ce77903c87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:23.149848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b8c92-ff74-4239-b298-2ce77903c87a None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:23.150131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b8c92-ff74-4239-b298-2ce77903c87a None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:23.150405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b8c92-ff74-4239-b298-2ce77903c87a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:23.157066 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b8c92-ff74-4239-b298-2ce77903c87a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:23.158511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b8c92-ff74-4239-b298-2ce77903c87a None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application [None req-4d8b8c92-ff74-4239-b298-2ce77903c87a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 07:10:23.171765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:23.172889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:23.174037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 07:10:23.174037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 07:10:23.174037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 07:10:23.174037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 07:10:23.174037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 07:10:23.174037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 07:10:23.174037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 07:10:23.174037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: ERROR keystone.server.flask.application Jun 17 07:10:23.174037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 638/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jun 17 07:10:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:23.178527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e324f1d-0615-4b05-b3a4-dc58d7bde99d None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:23.226335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:23.274486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e324f1d-0615-4b05-b3a4-dc58d7bde99d None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:23.307745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d531fa-7fc9-49eb-8f46-bcefbd1d3a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:23.308414 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d531fa-7fc9-49eb-8f46-bcefbd1d3a35 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:23.308704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d531fa-7fc9-49eb-8f46-bcefbd1d3a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:23.308985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d531fa-7fc9-49eb-8f46-bcefbd1d3a35 None admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:23.309764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d531fa-7fc9-49eb-8f46-bcefbd1d3a35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:23.310076 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d531fa-7fc9-49eb-8f46-bcefbd1d3a35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:23.311496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d531fa-7fc9-49eb-8f46-bcefbd1d3a35 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:23.323565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 648/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jun 17 07:10:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:23.329403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b8c92-ff74-4239-b298-2ce77903c87a None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:23.427120 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b8c92-ff74-4239-b298-2ce77903c87a None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:23.459852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb821f-45e0-4acf-a183-dd66a86bfcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:23.460543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb821f-45e0-4acf-a183-dd66a86bfcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:23.460830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb821f-45e0-4acf-a183-dd66a86bfcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:23.461110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb821f-45e0-4acf-a183-dd66a86bfcf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:23.461935 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb821f-45e0-4acf-a183-dd66a86bfcf0 None admin] RBAC: Adding query filter params (role.id=c625cb8707d040dfbdcf8f218c0a8287, scope.project.id=a4ec9b09302d4c918ef638d794468ba3, user.id=b8cf02a52d354517b363c0aaf477550c) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:23.462247 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb821f-45e0-4acf-a183-dd66a86bfcf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:23.463710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb821f-45e0-4acf-a183-dd66a86bfcf0 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:23.469514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 639/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jun 17 07:10:23 2022] GET /identity/v3/role_assignments?user.id=b8cf02a52d354517b363c0aaf477550c&scope.project.id=a4ec9b09302d4c918ef638d794468ba3&role.id=c625cb8707d040dfbdcf8f218c0a8287 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:25.558608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604844a-30b4-475e-98c5-6f08f57caa31 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:25.559131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604844a-30b4-475e-98c5-6f08f57caa31 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:25.559259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604844a-30b4-475e-98c5-6f08f57caa31 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:25.560568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 649/1288] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:10:25.565501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5ec6a-4294-4449-ba7d-4ad09934cebd None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:25.565913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5ec6a-4294-4449-ba7d-4ad09934cebd None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:25.566072 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5ec6a-4294-4449-ba7d-4ad09934cebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:25.582876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-dac5ec6a-4294-4449-ba7d-4ad09934cebd None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:10:25.584654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-dac5ec6a-4294-4449-ba7d-4ad09934cebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:10:25.612819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:25.649332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 640/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:10:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:10:25.671250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604844a-30b4-475e-98c5-6f08f57caa31 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:25.768504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604844a-30b4-475e-98c5-6f08f57caa31 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:25.778869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:25.801635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39b2ef-de69-43d5-b032-600426717cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:25.802929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39b2ef-de69-43d5-b032-600426717cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:25.803213 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39b2ef-de69-43d5-b032-600426717cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:25.803491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39b2ef-de69-43d5-b032-600426717cb2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:25.804160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39b2ef-de69-43d5-b032-600426717cb2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:25.805534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39b2ef-de69-43d5-b032-600426717cb2 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-6a39b2ef-de69-43d5-b032-600426717cb2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:25.817122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:25.818843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:25.819698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 650/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jun 17 07:10:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:25.824559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5ec6a-4294-4449-ba7d-4ad09934cebd None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:25.918563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5ec6a-4294-4449-ba7d-4ad09934cebd None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:25.951193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d71e8f-0b98-42f8-bca4-4f08d19b69b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:25.951843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d71e8f-0b98-42f8-bca4-4f08d19b69b4 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:25.952155 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d71e8f-0b98-42f8-bca4-4f08d19b69b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:25.952435 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d71e8f-0b98-42f8-bca4-4f08d19b69b4 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:25.953135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d71e8f-0b98-42f8-bca4-4f08d19b69b4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:25.953442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d71e8f-0b98-42f8-bca4-4f08d19b69b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:25.954797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d71e8f-0b98-42f8-bca4-4f08d19b69b4 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:25.959482 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 641/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Fri Jun 17 07:10:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:25.964924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39b2ef-de69-43d5-b032-600426717cb2 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:26.061778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39b2ef-de69-43d5-b032-600426717cb2 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:26.093915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa26f3c-3a81-4a72-9d93-2b4622bb3f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:26.094559 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa26f3c-3a81-4a72-9d93-2b4622bb3f65 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:26.094847 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa26f3c-3a81-4a72-9d93-2b4622bb3f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:26.095127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa26f3c-3a81-4a72-9d93-2b4622bb3f65 None admin] PATH_INFO: `/v3/services` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:26.095821 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa26f3c-3a81-4a72-9d93-2b4622bb3f65 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:26.096159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa26f3c-3a81-4a72-9d93-2b4622bb3f65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:26.097515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa26f3c-3a81-4a72-9d93-2b4622bb3f65 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:26.101832 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 651/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Fri Jun 17 07:10:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:28.218269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c73fbb-12cb-4442-a21f-885bd5ad443b None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:28.218805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c73fbb-12cb-4442-a21f-885bd5ad443b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:28.218805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c73fbb-12cb-4442-a21f-885bd5ad443b None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:28.220462 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 642/1293] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:10:28.225585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ece3e-b62c-48ee-9c7a-75b55720b5a9 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:28.226079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ece3e-b62c-48ee-9c7a-75b55720b5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:28.226227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ece3e-b62c-48ee-9c7a-75b55720b5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:28.243474 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-6b0ece3e-b62c-48ee-9c7a-75b55720b5a9 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:10:28.244721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6b0ece3e-b62c-48ee-9c7a-75b55720b5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:10:28.278557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:28.314374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 652/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:10:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:10:28.337340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c73fbb-12cb-4442-a21f-885bd5ad443b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:28.367141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:28.436726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c73fbb-12cb-4442-a21f-885bd5ad443b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:28.468641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85118148-7b58-46f3-8ebe-04fcd2fce8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:28.469241 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85118148-7b58-46f3-8ebe-04fcd2fce8a1 None admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:28.469527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85118148-7b58-46f3-8ebe-04fcd2fce8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:28.469806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85118148-7b58-46f3-8ebe-04fcd2fce8a1 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:28.470529 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85118148-7b58-46f3-8ebe-04fcd2fce8a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:28.471898 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85118148-7b58-46f3-8ebe-04fcd2fce8a1 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:28.484754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 643/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jun 17 07:10:28 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:10:30.556365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e973bb-8462-4e89-b68a-ba25fb019fbc None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:30.556905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e973bb-8462-4e89-b68a-ba25fb019fbc None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:30.556996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e973bb-8462-4e89-b68a-ba25fb019fbc None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:30.558300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 653/1296] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:10:30.563456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fc1f5-d259-496d-a1f8-857ddd9a5f00 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:30.563870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fc1f5-d259-496d-a1f8-857ddd9a5f00 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:30.564014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fc1f5-d259-496d-a1f8-857ddd9a5f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:30.580659 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-1c9fc1f5-d259-496d-a1f8-857ddd9a5f00 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:10:30.581827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1c9fc1f5-d259-496d-a1f8-857ddd9a5f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:10:30.651010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 644/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:10:30 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:10:30.673570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e973bb-8462-4e89-b68a-ba25fb019fbc None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:30.771064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e973bb-8462-4e89-b68a-ba25fb019fbc None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:30.804598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1b6a3-a7a8-4744-a76f-cfe888d799b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:30.805370 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1b6a3-a7a8-4744-a76f-cfe888d799b8 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:30.805656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1b6a3-a7a8-4744-a76f-cfe888d799b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:30.805933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1b6a3-a7a8-4744-a76f-cfe888d799b8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:30.806573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1b6a3-a7a8-4744-a76f-cfe888d799b8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:30.807926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1b6a3-a7a8-4744-a76f-cfe888d799b8 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-61d1b6a3-a7a8-4744-a76f-cfe888d799b8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:30.819536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:30.820753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:30.821676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 654/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jun 17 07:10:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:30.826329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fc1f5-d259-496d-a1f8-857ddd9a5f00 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:30.843656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:30.922489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fc1f5-d259-496d-a1f8-857ddd9a5f00 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:30.954691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccabfa-6be6-4b91-940c-f973d1e99e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:30.955344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccabfa-6be6-4b91-940c-f973d1e99e99 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:30.955631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccabfa-6be6-4b91-940c-f973d1e99e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:30.955910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccabfa-6be6-4b91-940c-f973d1e99e99 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:30.956643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccabfa-6be6-4b91-940c-f973d1e99e99 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:30.956952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccabfa-6be6-4b91-940c-f973d1e99e99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:30.958340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccabfa-6be6-4b91-940c-f973d1e99e99 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:30.963847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 645/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Fri Jun 17 07:10:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:30.969506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1b6a3-a7a8-4744-a76f-cfe888d799b8 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:31.000032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:31.065345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1b6a3-a7a8-4744-a76f-cfe888d799b8 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:31.097112 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc235d-d065-4bf5-a396-b29e9040c28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:31.097747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc235d-d065-4bf5-a396-b29e9040c28a None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:31.098034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc235d-d065-4bf5-a396-b29e9040c28a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:31.098311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc235d-d065-4bf5-a396-b29e9040c28a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:31.099085 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc235d-d065-4bf5-a396-b29e9040c28a None admin] RBAC: Adding query filter params (interface=public, service_id=a3fdc1993015440c8cc19ad7402678c3, region_id=RegionOne) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:31.099397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc235d-d065-4bf5-a396-b29e9040c28a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:31.100789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc235d-d065-4bf5-a396-b29e9040c28a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:31.105604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 655/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jun 17 07:10:30 2022] GET /identity/v3/endpoints?service_id=a3fdc1993015440c8cc19ad7402678c3&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:31.111035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccabfa-6be6-4b91-940c-f973d1e99e99 None admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:31.195360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:31.213865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccabfa-6be6-4b91-940c-f973d1e99e99 None admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:31.245942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54471af3-5596-45fb-a40a-1c0561564911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:31.246597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54471af3-5596-45fb-a40a-1c0561564911 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:31.246881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54471af3-5596-45fb-a40a-1c0561564911 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:31.247158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54471af3-5596-45fb-a40a-1c0561564911 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:31.247822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54471af3-5596-45fb-a40a-1c0561564911 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:31.249220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54471af3-5596-45fb-a40a-1c0561564911 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:31.255769 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 646/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jun 17 07:10:31 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:10:33.304299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a8293-3061-4c86-b3fa-7a607dafc94e None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:33.304857 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a8293-3061-4c86-b3fa-7a607dafc94e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:33.304857 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a8293-3061-4c86-b3fa-7a607dafc94e None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:33.306220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 656/1302] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:10:33.311118 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bacfbe-c93a-4b66-b83a-04b48a7124a4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:33.311547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bacfbe-c93a-4b66-b83a-04b48a7124a4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:33.311675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bacfbe-c93a-4b66-b83a-04b48a7124a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:33.328548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-47bacfbe-c93a-4b66-b83a-04b48a7124a4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:10:33.329729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-47bacfbe-c93a-4b66-b83a-04b48a7124a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:10:33.394917 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 647/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:10:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 07:10:33.416973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a8293-3061-4c86-b3fa-7a607dafc94e None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:33.510153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:33.516925 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a8293-3061-4c86-b3fa-7a607dafc94e None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:33.549393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345724ed-c02b-4a89-95c1-252b974e7399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:33.550174 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345724ed-c02b-4a89-95c1-252b974e7399 None admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:33.550466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345724ed-c02b-4a89-95c1-252b974e7399 None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:33.550748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345724ed-c02b-4a89-95c1-252b974e7399 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:33.551397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345724ed-c02b-4a89-95c1-252b974e7399 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:33.553324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345724ed-c02b-4a89-95c1-252b974e7399 None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application [None req-345724ed-c02b-4a89-95c1-252b974e7399 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 07:10:33.565083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 07:10:33.566255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application with Lock( Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self._enter() Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application return self.creator() Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application created_value = creator( Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: ERROR keystone.server.flask.application Jun 17 07:10:33.567264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 657/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jun 17 07:10:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 07:10:33.572656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bacfbe-c93a-4b66-b83a-04b48a7124a4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:33.644507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:33.667727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bacfbe-c93a-4b66-b83a-04b48a7124a4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:33.700230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640dc67e-3003-4d3a-a1cf-5eda89763175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:33.700886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640dc67e-3003-4d3a-a1cf-5eda89763175 None admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:33.701173 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640dc67e-3003-4d3a-a1cf-5eda89763175 None admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:33.701452 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640dc67e-3003-4d3a-a1cf-5eda89763175 None admin] PATH_INFO: `/v3/services` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:33.702148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640dc67e-3003-4d3a-a1cf-5eda89763175 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:10:33.702455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640dc67e-3003-4d3a-a1cf-5eda89763175 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:33.703850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640dc67e-3003-4d3a-a1cf-5eda89763175 None admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:33.708935 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 648/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Fri Jun 17 07:10:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:10:33.714671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345724ed-c02b-4a89-95c1-252b974e7399 None admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:10:33.811758 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345724ed-c02b-4a89-95c1-252b974e7399 None admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:10:33.843749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a509e5a-0ab6-4e92-9f35-8a3aee78b02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:10:33.844361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a509e5a-0ab6-4e92-9f35-8a3aee78b02a None admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:10:33.844652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a509e5a-0ab6-4e92-9f35-8a3aee78b02a None admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:10:33.844934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a509e5a-0ab6-4e92-9f35-8a3aee78b02a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:10:33.845671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a509e5a-0ab6-4e92-9f35-8a3aee78b02a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:10:33.847021 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a509e5a-0ab6-4e92-9f35-8a3aee78b02a None admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:10:33.865448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 658/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jun 17 07:10:33 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:10:43.706253 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=30 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:10:43.860437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:35.318152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91553cd2-d60e-4a7b-bbf5-57c3b934e600 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:35.318762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91553cd2-d60e-4a7b-bbf5-57c3b934e600 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:35.318762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91553cd2-d60e-4a7b-bbf5-57c3b934e600 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:35.320233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 649/1307] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:11:35.325145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d5b58-a4e6-4605-8bda-7fc5e0d28413 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:35.325579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d5b58-a4e6-4605-8bda-7fc5e0d28413 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:35.325791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d5b58-a4e6-4605-8bda-7fc5e0d28413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:35.450607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f61d5b58-a4e6-4605-8bda-7fc5e0d28413 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:11:35.451826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f61d5b58-a4e6-4605-8bda-7fc5e0d28413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:35.647593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 659/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:11:35.700267 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e7ba4-cf07-43b7-8823-44c6d439931c None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:35.700551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e7ba4-cf07-43b7-8823-44c6d439931c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:35.700851 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e7ba4-cf07-43b7-8823-44c6d439931c None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:35.702222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 650/1309] 10.222.0.33 () {54 vars in 964 bytes} [Fri Jun 17 07:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:11:35.708396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a7478-512a-4dd9-92b5-65b6757bb437 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:35.709291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a7478-512a-4dd9-92b5-65b6757bb437 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:35.709572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a7478-512a-4dd9-92b5-65b6757bb437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:35.790204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-783a7478-512a-4dd9-92b5-65b6757bb437 None None] MFA Rules not processed for user `7e0797679c07454abc867cd7b30c8b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:11:35.791333 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-783a7478-512a-4dd9-92b5-65b6757bb437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:35.812301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:35.873096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 660/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Fri Jun 17 07:11:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:11:35.882516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e7ba4-cf07-43b7-8823-44c6d439931c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:11:35.884368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d53e7ba4-cf07-43b7-8823-44c6d439931c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:35.993340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e7ba4-cf07-43b7-8823-44c6d439931c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:11:36.069643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc02cff-d1b5-49ae-a166-927c2f53f4f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:11:36.070322 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc02cff-d1b5-49ae-a166-927c2f53f4f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:36.070607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc02cff-d1b5-49ae-a166-927c2f53f4f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:36.070879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc02cff-d1b5-49ae-a166-927c2f53f4f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:36.072676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5cc02cff-d1b5-49ae-a166-927c2f53f4f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:36.130842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:36.199768 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc02cff-d1b5-49ae-a166-927c2f53f4f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:11:36.201336 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc02cff-d1b5-49ae-a166-927c2f53f4f6 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:11:36.224234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 651/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:11:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:11:38.635724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496f48b-699d-47c5-a448-b5c68ea9e15f None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:38.636281 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496f48b-699d-47c5-a448-b5c68ea9e15f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:38.636374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496f48b-699d-47c5-a448-b5c68ea9e15f None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:38.637788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 661/1312] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:11:38.643132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136171ec-ab6b-46b6-9b43-1dc34ddaf7ef None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:38.643559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136171ec-ab6b-46b6-9b43-1dc34ddaf7ef None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:38.643725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136171ec-ab6b-46b6-9b43-1dc34ddaf7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:38.716131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-136171ec-ab6b-46b6-9b43-1dc34ddaf7ef None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:11:38.717366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-136171ec-ab6b-46b6-9b43-1dc34ddaf7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:38.820661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 652/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:11:38.859384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496f48b-699d-47c5-a448-b5c68ea9e15f None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:11:38.860955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7496f48b-699d-47c5-a448-b5c68ea9e15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:38.970973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496f48b-699d-47c5-a448-b5c68ea9e15f None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:11:39.058145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:39.060974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04554e3d-8ca1-4164-a732-d1096935bd8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:11:39.061806 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04554e3d-8ca1-4164-a732-d1096935bd8c service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:39.062515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04554e3d-8ca1-4164-a732-d1096935bd8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:39.063182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04554e3d-8ca1-4164-a732-d1096935bd8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:39.065482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-04554e3d-8ca1-4164-a732-d1096935bd8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:39.185939 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04554e3d-8ca1-4164-a732-d1096935bd8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:11:39.187389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04554e3d-8ca1-4164-a732-d1096935bd8c service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:11:39.209680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 662/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:11:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:11:41.665508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab10108-8d9e-4715-a789-493ee9b7b88b None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:41.666013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab10108-8d9e-4715-a789-493ee9b7b88b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:41.666013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab10108-8d9e-4715-a789-493ee9b7b88b None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:41.667348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 653/1315] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:11:41.672471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844e79b-e855-4b89-a3ed-d43476755fe2 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:41.672889 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844e79b-e855-4b89-a3ed-d43476755fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:41.673028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844e79b-e855-4b89-a3ed-d43476755fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:41.735999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-e844e79b-e855-4b89-a3ed-d43476755fe2 None None] MFA Rules not processed for user `8fb531e37b334e7ea25ec28ef9d7d181`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:11:41.737224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e844e79b-e855-4b89-a3ed-d43476755fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:41.787104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:41.831038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 663/1316] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:11:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:11:41.870107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab10108-8d9e-4715-a789-493ee9b7b88b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:11:41.871643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cab10108-8d9e-4715-a789-493ee9b7b88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:41.881855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:41.974724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab10108-8d9e-4715-a789-493ee9b7b88b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:11:42.045757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bdf2d-0f59-4465-9249-17d6ff38e6bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:11:42.046406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdf2d-0f59-4465-9249-17d6ff38e6bf service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:42.046692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdf2d-0f59-4465-9249-17d6ff38e6bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:42.046972 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdf2d-0f59-4465-9249-17d6ff38e6bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:42.048659 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7f6bdf2d-0f59-4465-9249-17d6ff38e6bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:42.108850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:42.154937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bdf2d-0f59-4465-9249-17d6ff38e6bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:11:42.156661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bdf2d-0f59-4465-9249-17d6ff38e6bf service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:11:42.176248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 654/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:11:41 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:11:44.803043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5deae-ac2d-49cd-a09b-a0af4389b3bb None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:44.803577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5deae-ac2d-49cd-a09b-a0af4389b3bb None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:44.803577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5deae-ac2d-49cd-a09b-a0af4389b3bb None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:44.805040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 664/1318] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:11:44.810127 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f870d0-76c3-408c-8271-86d86f2bcc85 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:44.810556 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f870d0-76c3-408c-8271-86d86f2bcc85 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:44.810674 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f870d0-76c3-408c-8271-86d86f2bcc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:44.873446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-b5f870d0-76c3-408c-8271-86d86f2bcc85 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:11:44.874664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b5f870d0-76c3-408c-8271-86d86f2bcc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:44.968628 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 655/1319] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:11:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:11:45.006645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5deae-ac2d-49cd-a09b-a0af4389b3bb None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:11:45.008223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9db5deae-ac2d-49cd-a09b-a0af4389b3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:45.108412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5deae-ac2d-49cd-a09b-a0af4389b3bb None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:11:45.178866 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61041c83-af68-49d5-a646-af43ead99af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:11:45.179524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61041c83-af68-49d5-a646-af43ead99af8 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:45.179805 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61041c83-af68-49d5-a646-af43ead99af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:45.180103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61041c83-af68-49d5-a646-af43ead99af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:45.181695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-61041c83-af68-49d5-a646-af43ead99af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:45.188803 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:45.289583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61041c83-af68-49d5-a646-af43ead99af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:11:45.291007 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61041c83-af68-49d5-a646-af43ead99af8 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:11:45.309389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 665/1320] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:11:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:11:48.062994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb78cb6-4a90-44af-a6de-61bd15281a7e None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:48.063543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb78cb6-4a90-44af-a6de-61bd15281a7e None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:48.063543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb78cb6-4a90-44af-a6de-61bd15281a7e None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:48.064962 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 656/1321] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:11:48.070640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328265dd-b1c6-4775-9154-30dce8d42026 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:48.071074 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328265dd-b1c6-4775-9154-30dce8d42026 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:48.071269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328265dd-b1c6-4775-9154-30dce8d42026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:48.142706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-328265dd-b1c6-4775-9154-30dce8d42026 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:11:48.143953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-328265dd-b1c6-4775-9154-30dce8d42026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:48.210843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:48.252527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 666/1322] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:11:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:11:48.289138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb78cb6-4a90-44af-a6de-61bd15281a7e None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:11:48.290710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8cb78cb6-4a90-44af-a6de-61bd15281a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:48.310216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:48.397455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb78cb6-4a90-44af-a6de-61bd15281a7e None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:11:48.468460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecad47-fbb4-429f-b14b-33fb876ea0f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:11:48.469104 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecad47-fbb4-429f-b14b-33fb876ea0f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:48.469386 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecad47-fbb4-429f-b14b-33fb876ea0f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:48.469661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecad47-fbb4-429f-b14b-33fb876ea0f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:48.471278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8aecad47-fbb4-429f-b14b-33fb876ea0f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:48.541252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:48.579154 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecad47-fbb4-429f-b14b-33fb876ea0f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:11:48.580636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecad47-fbb4-429f-b14b-33fb876ea0f3 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:11:48.600371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 657/1323] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:11:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:11:50.785136 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe71d6-657f-4d3d-8925-54c9cb714f2e None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:50.785735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe71d6-657f-4d3d-8925-54c9cb714f2e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:50.785735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe71d6-657f-4d3d-8925-54c9cb714f2e None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:50.787128 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 667/1324] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:11:50.792285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73013751-e820-410d-b73f-04fb4e4560fd None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:50.792683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73013751-e820-410d-b73f-04fb4e4560fd None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:50.792851 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73013751-e820-410d-b73f-04fb4e4560fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:50.857457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-73013751-e820-410d-b73f-04fb4e4560fd None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:11:50.858658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-73013751-e820-410d-b73f-04fb4e4560fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:50.953407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 658/1325] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:11:50.988114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe71d6-657f-4d3d-8925-54c9cb714f2e None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:11:50.989669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-0dfe71d6-657f-4d3d-8925-54c9cb714f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:51.096198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe71d6-657f-4d3d-8925-54c9cb714f2e None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:11:51.168562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8bc48-fcee-49cb-866e-2ff4f1890d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:11:51.169213 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8bc48-fcee-49cb-866e-2ff4f1890d94 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:51.169490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8bc48-fcee-49cb-866e-2ff4f1890d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:51.169772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8bc48-fcee-49cb-866e-2ff4f1890d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:51.171426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-65d8bc48-fcee-49cb-866e-2ff4f1890d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:51.188505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:51.292890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8bc48-fcee-49cb-866e-2ff4f1890d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:11:51.294569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8bc48-fcee-49cb-866e-2ff4f1890d94 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:11:51.313507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 668/1326] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:11:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:11:53.484163 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94701aeb-8839-4a18-b2bb-f6c70f98e208 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:53.484606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94701aeb-8839-4a18-b2bb-f6c70f98e208 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:53.484728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94701aeb-8839-4a18-b2bb-f6c70f98e208 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:53.486113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 659/1327] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:11:53.491698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e970f1-812e-4aa9-bd29-2f5d40eac640 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:53.492120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e970f1-812e-4aa9-bd29-2f5d40eac640 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:53.492301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e970f1-812e-4aa9-bd29-2f5d40eac640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:53.555767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-17e970f1-812e-4aa9-bd29-2f5d40eac640 None None] MFA Rules not processed for user `8fb531e37b334e7ea25ec28ef9d7d181`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:11:53.557004 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-17e970f1-812e-4aa9-bd29-2f5d40eac640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:53.632369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:53.665857 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 669/1328] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:11:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:11:53.714806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b4193-e96c-4b01-9b83-109982ab78a0 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:53.715092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b4193-e96c-4b01-9b83-109982ab78a0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:53.715372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b4193-e96c-4b01-9b83-109982ab78a0 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:53.716775 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 660/1329] 10.222.0.33 () {54 vars in 964 bytes} [Fri Jun 17 07:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:11:53.722863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60c987-8e19-46b0-86ca-c4e71a6ec8c3 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:53.723031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60c987-8e19-46b0-86ca-c4e71a6ec8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:53.723316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60c987-8e19-46b0-86ca-c4e71a6ec8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:53.811807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ed60c987-8e19-46b0-86ca-c4e71a6ec8c3 None None] MFA Rules not processed for user `7e0797679c07454abc867cd7b30c8b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:11:53.813083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ed60c987-8e19-46b0-86ca-c4e71a6ec8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:53.891670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 670/1330] 10.222.0.33 () {58 vars in 1061 bytes} [Fri Jun 17 07:11:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:11:53.900608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b4193-e96c-4b01-9b83-109982ab78a0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:11:53.902130 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ec8b4193-e96c-4b01-9b83-109982ab78a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:53.933316 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:54.004124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b4193-e96c-4b01-9b83-109982ab78a0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:11:54.076957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d2a0d-29ec-4490-8400-0e619427ca10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:11:54.077608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2a0d-29ec-4490-8400-0e619427ca10 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:54.077887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2a0d-29ec-4490-8400-0e619427ca10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:54.078171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2a0d-29ec-4490-8400-0e619427ca10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:54.079801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-df1d2a0d-29ec-4490-8400-0e619427ca10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:54.155583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:54.188372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d2a0d-29ec-4490-8400-0e619427ca10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:11:54.189837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d2a0d-29ec-4490-8400-0e619427ca10 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:11:54.207191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 661/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:11:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:11:56.942240 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcc0ce-de73-4a31-9daa-a7af7767e223 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:56.942850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcc0ce-de73-4a31-9daa-a7af7767e223 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:56.942850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcc0ce-de73-4a31-9daa-a7af7767e223 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:56.944237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 671/1332] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:11:56.949456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fe866-eb53-40a4-a5c3-a7129765d370 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:56.949876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fe866-eb53-40a4-a5c3-a7129765d370 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:56.950027 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fe866-eb53-40a4-a5c3-a7129765d370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:57.012611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-ff2fe866-eb53-40a4-a5c3-a7129765d370 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:11:57.013811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ff2fe866-eb53-40a4-a5c3-a7129765d370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:57.108145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 662/1333] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:11:57.143350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcc0ce-de73-4a31-9daa-a7af7767e223 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:11:57.144910 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-84bcc0ce-de73-4a31-9daa-a7af7767e223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:57.212972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:57.246700 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcc0ce-de73-4a31-9daa-a7af7767e223 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:11:57.317495 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494f784-61c4-4897-966f-a3b0c07a57bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:11:57.318727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494f784-61c4-4897-966f-a3b0c07a57bc service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:57.319011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494f784-61c4-4897-966f-a3b0c07a57bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:57.319284 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494f784-61c4-4897-966f-a3b0c07a57bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:57.320947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a494f784-61c4-4897-966f-a3b0c07a57bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:57.426839 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494f784-61c4-4897-966f-a3b0c07a57bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:11:57.428839 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494f784-61c4-4897-966f-a3b0c07a57bc service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:11:57.438429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:57.447660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 672/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:11:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:11:59.585278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33d525-167c-472a-b2d1-0b9240848227 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:59.585828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33d525-167c-472a-b2d1-0b9240848227 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:59.585828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33d525-167c-472a-b2d1-0b9240848227 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:59.587239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 663/1335] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:11:59.592416 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d209f-d316-4fff-8505-ba7c2f61177f None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:59.592837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d209f-d316-4fff-8505-ba7c2f61177f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:59.592979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d209f-d316-4fff-8505-ba7c2f61177f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:59.657473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-aa1d209f-d316-4fff-8505-ba7c2f61177f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:11:59.658619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-aa1d209f-d316-4fff-8505-ba7c2f61177f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:59.751892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 673/1336] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:11:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:11:59.792182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33d525-167c-472a-b2d1-0b9240848227 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:11:59.793718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-2f33d525-167c-472a-b2d1-0b9240848227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:11:59.832022 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:11:59.894875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33d525-167c-472a-b2d1-0b9240848227 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:11:59.965505 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120a2d6-c21a-4a44-aa49-b07145ec8fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:11:59.966147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120a2d6-c21a-4a44-aa49-b07145ec8fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:11:59.966432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120a2d6-c21a-4a44-aa49-b07145ec8fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:11:59.966709 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120a2d6-c21a-4a44-aa49-b07145ec8fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:11:59.968358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8120a2d6-c21a-4a44-aa49-b07145ec8fb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:00.048080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:12:00.075859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8120a2d6-c21a-4a44-aa49-b07145ec8fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:00.077320 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8120a2d6-c21a-4a44-aa49-b07145ec8fb5 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:00.095145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 664/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:11:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:02.635222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e3e9e-883b-462f-b591-818d64679f24 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:02.635755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e3e9e-883b-462f-b591-818d64679f24 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:02.635755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e3e9e-883b-462f-b591-818d64679f24 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:02.637225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 674/1338] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:02.642170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670f652-af81-41b7-a5dc-15d655da671b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:02.642603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670f652-af81-41b7-a5dc-15d655da671b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:02.642734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670f652-af81-41b7-a5dc-15d655da671b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:02.706249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-4670f652-af81-41b7-a5dc-15d655da671b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:02.707438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-4670f652-af81-41b7-a5dc-15d655da671b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:02.799857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 665/1339] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:02.835377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e3e9e-883b-462f-b591-818d64679f24 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:02.836934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b77e3e9e-883b-462f-b591-818d64679f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:02.910531 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:12:02.939101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e3e9e-883b-462f-b591-818d64679f24 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:03.010011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba279a-fdc8-4c53-9080-42f71b9a7a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:03.010649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba279a-fdc8-4c53-9080-42f71b9a7a56 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:03.010936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba279a-fdc8-4c53-9080-42f71b9a7a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:03.011216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba279a-fdc8-4c53-9080-42f71b9a7a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:03.012889 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-daba279a-fdc8-4c53-9080-42f71b9a7a56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:03.117228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba279a-fdc8-4c53-9080-42f71b9a7a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:03.118655 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba279a-fdc8-4c53-9080-42f71b9a7a56 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:03.136946 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 675/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:07.177808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7c0fa-a76b-461a-bc9e-5d6241aabdaf None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:07.178453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7c0fa-a76b-461a-bc9e-5d6241aabdaf None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:07.178453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7c0fa-a76b-461a-bc9e-5d6241aabdaf None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:07.179691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 666/1341] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:07.186428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0ab2b-10e2-49f0-9417-a676639eee58 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:07.186428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0ab2b-10e2-49f0-9417-a676639eee58 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:07.186428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0ab2b-10e2-49f0-9417-a676639eee58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:07.194998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:12:07.270354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-82f0ab2b-10e2-49f0-9417-a676639eee58 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:07.271984 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-82f0ab2b-10e2-49f0-9417-a676639eee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:07.389583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 676/1342] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:07.428735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7c0fa-a76b-461a-bc9e-5d6241aabdaf None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:07.430867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-18a7c0fa-a76b-461a-bc9e-5d6241aabdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:07.481866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:12:07.530571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7c0fa-a76b-461a-bc9e-5d6241aabdaf None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:07.534451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909f109-066b-4323-ad35-9f066261ffc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:07.535100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909f109-066b-4323-ad35-9f066261ffc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:07.535384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909f109-066b-4323-ad35-9f066261ffc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:07.535664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909f109-066b-4323-ad35-9f066261ffc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:07.545285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909f109-066b-4323-ad35-9f066261ffc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:07.546733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909f109-066b-4323-ad35-9f066261ffc7 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:07.556398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 667/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:10.268810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d865245-52aa-4aee-812f-a8da69d9373b None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:10.269406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d865245-52aa-4aee-812f-a8da69d9373b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:10.269406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d865245-52aa-4aee-812f-a8da69d9373b None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:10.270755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 677/1344] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:10.276568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7e1f8-6552-45a0-96a9-bcd3fd727499 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:10.276990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7e1f8-6552-45a0-96a9-bcd3fd727499 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:10.277141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7e1f8-6552-45a0-96a9-bcd3fd727499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:10.307440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-24c7e1f8-6552-45a0-96a9-bcd3fd727499 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:10.308714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-24c7e1f8-6552-45a0-96a9-bcd3fd727499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:10.392642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 668/1345] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:10.430204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d865245-52aa-4aee-812f-a8da69d9373b None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:10.431932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-0d865245-52aa-4aee-812f-a8da69d9373b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:10.446024 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d865245-52aa-4aee-812f-a8da69d9373b None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:10.449524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce27d6-b8e3-4e80-858b-0605306312b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:10.450166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce27d6-b8e3-4e80-858b-0605306312b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:10.450446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce27d6-b8e3-4e80-858b-0605306312b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:10.450719 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce27d6-b8e3-4e80-858b-0605306312b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:10.460355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce27d6-b8e3-4e80-858b-0605306312b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:10.461789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce27d6-b8e3-4e80-858b-0605306312b7 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:10.471478 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 678/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:14.815784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbec235-7b77-4cb4-9254-cc238f6293a7 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:14.816352 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbec235-7b77-4cb4-9254-cc238f6293a7 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:14.816480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbec235-7b77-4cb4-9254-cc238f6293a7 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:14.817842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 669/1347] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:14.823385 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfa1fb-d350-4d75-bc2d-145658e21580 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:14.823783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfa1fb-d350-4d75-bc2d-145658e21580 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:14.823947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfa1fb-d350-4d75-bc2d-145658e21580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:14.842123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-06dfa1fb-d350-4d75-bc2d-145658e21580 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:14.843451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-06dfa1fb-d350-4d75-bc2d-145658e21580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:14.852426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 679/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:14.890159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbec235-7b77-4cb4-9254-cc238f6293a7 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:14.901518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbec235-7b77-4cb4-9254-cc238f6293a7 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:14.905039 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1adf8-4c67-46eb-a497-c40114a8e62d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:14.905683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1adf8-4c67-46eb-a497-c40114a8e62d service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:14.905971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1adf8-4c67-46eb-a497-c40114a8e62d service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:14.906252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1adf8-4c67-46eb-a497-c40114a8e62d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:14.915542 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1adf8-4c67-46eb-a497-c40114a8e62d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:14.917015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1adf8-4c67-46eb-a497-c40114a8e62d service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:14.926954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 670/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:19.961107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909da42-bf33-4761-b25a-020f42ae7d74 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:19.961649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909da42-bf33-4761-b25a-020f42ae7d74 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:19.961875 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909da42-bf33-4761-b25a-020f42ae7d74 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:19.963666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 680/1350] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:19.969123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d437f-8857-422d-be46-ea0ad0211ffe None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:19.969876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d437f-8857-422d-be46-ea0ad0211ffe None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:19.970396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d437f-8857-422d-be46-ea0ad0211ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:20.044760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-2e8d437f-8857-422d-be46-ea0ad0211ffe None None] MFA Rules not processed for user `b8cf02a52d354517b363c0aaf477550c`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:20.046506 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-2e8d437f-8857-422d-be46-ea0ad0211ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:20.085724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:12:20.137697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 671/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:12:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:24.138841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf2158-21c2-43a1-8a21-951555849854 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:24.139470 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf2158-21c2-43a1-8a21-951555849854 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:24.139470 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf2158-21c2-43a1-8a21-951555849854 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:24.140994 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 681/1352] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:24.146015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80460add-5045-4024-bbb8-b8d529535468 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:24.146479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80460add-5045-4024-bbb8-b8d529535468 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:24.146613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80460add-5045-4024-bbb8-b8d529535468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:24.165982 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-80460add-5045-4024-bbb8-b8d529535468 None None] MFA Rules not processed for user `b8cf02a52d354517b363c0aaf477550c`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:24.167184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-80460add-5045-4024-bbb8-b8d529535468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:24.175998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 672/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:12:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:24.837218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=72 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:12:25.915653 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13707e93-62eb-4d4c-b672-dac47e3724f1 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:25.916331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13707e93-62eb-4d4c-b672-dac47e3724f1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:25.916331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13707e93-62eb-4d4c-b672-dac47e3724f1 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:25.917789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 682/1354] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:25.922539 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c209a-9bc6-44d0-8fad-2f849031b6f0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:25.922986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c209a-9bc6-44d0-8fad-2f849031b6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:25.923191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c209a-9bc6-44d0-8fad-2f849031b6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:25.941408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-d27c209a-9bc6-44d0-8fad-2f849031b6f0 None None] MFA Rules not processed for user `b8cf02a52d354517b363c0aaf477550c`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:25.942618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d27c209a-9bc6-44d0-8fad-2f849031b6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:25.951703 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 673/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:12:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:29.551263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf938c3-9fb9-43dd-acfe-4f4291178eff None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:29.551811 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf938c3-9fb9-43dd-acfe-4f4291178eff None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:29.552208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf938c3-9fb9-43dd-acfe-4f4291178eff None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:29.554135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 683/1356] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:29.559502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229768a3-0d3e-4d70-a399-d7b9f8c304e6 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:29.559978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229768a3-0d3e-4d70-a399-d7b9f8c304e6 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:29.560178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229768a3-0d3e-4d70-a399-d7b9f8c304e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:29.578527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-229768a3-0d3e-4d70-a399-d7b9f8c304e6 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:29.579653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-229768a3-0d3e-4d70-a399-d7b9f8c304e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:29.587752 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 674/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:29.640618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b5d85-4730-4168-bcfa-c7049115232f None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:29.641341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b5d85-4730-4168-bcfa-c7049115232f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:29.641842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b5d85-4730-4168-bcfa-c7049115232f None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:29.643721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 684/1358] 10.222.0.33 () {54 vars in 961 bytes} [Fri Jun 17 07:12:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:29.651664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19edea6-5cad-47ed-bf74-89482dc7cd31 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:29.652275 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19edea6-5cad-47ed-bf74-89482dc7cd31 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:29.652837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19edea6-5cad-47ed-bf74-89482dc7cd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:29.707296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c19edea6-5cad-47ed-bf74-89482dc7cd31 None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:29.708680 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c19edea6-5cad-47ed-bf74-89482dc7cd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:29.805937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 675/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Fri Jun 17 07:12:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:12:29.807079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b5d85-4730-4168-bcfa-c7049115232f None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:29.827160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b5d85-4730-4168-bcfa-c7049115232f None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:29.834962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5926528-2a3c-494d-89d6-8c02bf94fd42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:29.835730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5926528-2a3c-494d-89d6-8c02bf94fd42 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:29.836135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5926528-2a3c-494d-89d6-8c02bf94fd42 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:29.836490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5926528-2a3c-494d-89d6-8c02bf94fd42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:29.845973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5926528-2a3c-494d-89d6-8c02bf94fd42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:29.847663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5926528-2a3c-494d-89d6-8c02bf94fd42 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:29.859040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 685/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:12:32.191814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f066b-b7d5-4b95-9baf-aadb62aee910 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:32.192456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f066b-b7d5-4b95-9baf-aadb62aee910 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:32.192456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f066b-b7d5-4b95-9baf-aadb62aee910 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:32.194759 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 676/1361] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:32.200503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a6480-e9eb-4a0a-8134-3d9a992e7c1b None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:32.201039 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a6480-e9eb-4a0a-8134-3d9a992e7c1b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:32.201207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a6480-e9eb-4a0a-8134-3d9a992e7c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:32.220958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c99a6480-e9eb-4a0a-8134-3d9a992e7c1b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:32.222216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c99a6480-e9eb-4a0a-8134-3d9a992e7c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:32.231687 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 686/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:32.278570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f066b-b7d5-4b95-9baf-aadb62aee910 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:32.293868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f066b-b7d5-4b95-9baf-aadb62aee910 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:32.299618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb46d9-882d-4d6b-a0f7-9fca74537856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:32.301226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb46d9-882d-4d6b-a0f7-9fca74537856 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:32.302808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb46d9-882d-4d6b-a0f7-9fca74537856 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:32.303416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb46d9-882d-4d6b-a0f7-9fca74537856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:32.312902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb46d9-882d-4d6b-a0f7-9fca74537856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:32.314512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb46d9-882d-4d6b-a0f7-9fca74537856 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:32.325989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 677/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:34.788488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1465b-ebd4-406b-8485-92f0c9977397 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:34.795273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1465b-ebd4-406b-8485-92f0c9977397 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:34.795602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1465b-ebd4-406b-8485-92f0c9977397 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:34.797097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 687/1364] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:34.802373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb879f5-06a8-48f0-aa05-1ab7e9979c9e None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:34.803283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb879f5-06a8-48f0-aa05-1ab7e9979c9e None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:34.803580 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb879f5-06a8-48f0-aa05-1ab7e9979c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:34.822994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-4bb879f5-06a8-48f0-aa05-1ab7e9979c9e None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:34.824253 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-4bb879f5-06a8-48f0-aa05-1ab7e9979c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:34.832128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 678/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:34.884650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d123e5-e6ec-48e7-a5c4-508c9bb3a99b None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:34.884650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d123e5-e6ec-48e7-a5c4-508c9bb3a99b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:34.884650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d123e5-e6ec-48e7-a5c4-508c9bb3a99b None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:34.884650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 688/1366] 10.222.0.33 () {54 vars in 961 bytes} [Fri Jun 17 07:12:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:34.892026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05893799-7f7d-40de-8657-817ce9f9458c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:34.892393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05893799-7f7d-40de-8657-817ce9f9458c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:34.892685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05893799-7f7d-40de-8657-817ce9f9458c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:34.911368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-05893799-7f7d-40de-8657-817ce9f9458c None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:34.912595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-05893799-7f7d-40de-8657-817ce9f9458c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:34.922398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 679/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Fri Jun 17 07:12:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:34.930811 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d123e5-e6ec-48e7-a5c4-508c9bb3a99b None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:34.947960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d123e5-e6ec-48e7-a5c4-508c9bb3a99b None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:34.960859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbc530-76a2-4692-8714-0cba8c0ad20e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:34.961776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbc530-76a2-4692-8714-0cba8c0ad20e service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:34.962239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbc530-76a2-4692-8714-0cba8c0ad20e service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:34.962687 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbc530-76a2-4692-8714-0cba8c0ad20e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:34.973590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbc530-76a2-4692-8714-0cba8c0ad20e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:34.975633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbc530-76a2-4692-8714-0cba8c0ad20e service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:34.987954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 689/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:37.175121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1c73c-98fa-42c2-ab63-5969de17c72e None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:37.175658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1c73c-98fa-42c2-ab63-5969de17c72e None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:37.175658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1c73c-98fa-42c2-ab63-5969de17c72e None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:37.177257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 680/1369] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:37.182735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355194cf-f3cf-41bc-a35a-65d23ba6ae03 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:37.183158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355194cf-f3cf-41bc-a35a-65d23ba6ae03 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:37.183302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355194cf-f3cf-41bc-a35a-65d23ba6ae03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:37.205642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-355194cf-f3cf-41bc-a35a-65d23ba6ae03 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:37.207127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-355194cf-f3cf-41bc-a35a-65d23ba6ae03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:37.224167 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 690/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:37.261787 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1c73c-98fa-42c2-ab63-5969de17c72e None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:37.277408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1c73c-98fa-42c2-ab63-5969de17c72e None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:37.283208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cb2c7-17e9-4766-8f52-4d88607cb680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:37.283931 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cb2c7-17e9-4766-8f52-4d88607cb680 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:37.284330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cb2c7-17e9-4766-8f52-4d88607cb680 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:37.284682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cb2c7-17e9-4766-8f52-4d88607cb680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:37.293949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36cb2c7-17e9-4766-8f52-4d88607cb680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:37.295634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36cb2c7-17e9-4766-8f52-4d88607cb680 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:37.305358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 681/1371] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:37.615664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60367c43-672b-4e96-a641-515aa38cbced None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:37.616236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60367c43-672b-4e96-a641-515aa38cbced None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:37.616520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60367c43-672b-4e96-a641-515aa38cbced None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:37.617882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 691/1372] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:37.623302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f37fcb-600c-4ba8-8c32-68f7a0e8b32f None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:37.623666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f37fcb-600c-4ba8-8c32-68f7a0e8b32f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:37.624021 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f37fcb-600c-4ba8-8c32-68f7a0e8b32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:37.644035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-e5f37fcb-600c-4ba8-8c32-68f7a0e8b32f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:37.645274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e5f37fcb-600c-4ba8-8c32-68f7a0e8b32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:37.654971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 682/1373] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:12:37.968028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cf468-ec8a-4322-8554-dd384d1a5eab None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:37.968361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cf468-ec8a-4322-8554-dd384d1a5eab None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:37.968644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cf468-ec8a-4322-8554-dd384d1a5eab None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:37.970299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 692/1374] 10.222.0.33 () {54 vars in 963 bytes} [Fri Jun 17 07:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:37.975336 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f745a89-2a09-4813-86b1-958e131988eb None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:37.975697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f745a89-2a09-4813-86b1-958e131988eb None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:37.976047 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f745a89-2a09-4813-86b1-958e131988eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:37.984166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:12:38.030217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-5f745a89-2a09-4813-86b1-958e131988eb None None] MFA Rules not processed for user `3769534c610b448db693f7c5d6133e65`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:38.031891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5f745a89-2a09-4813-86b1-958e131988eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:38.101344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 683/1375] 10.222.0.33 () {58 vars in 1060 bytes} [Fri Jun 17 07:12:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:12:38.109034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cf468-ec8a-4322-8554-dd384d1a5eab None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:38.120612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cf468-ec8a-4322-8554-dd384d1a5eab None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:38.124109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f2d02-fc9f-4c20-9f61-a2b60b10f3d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:38.124752 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f2d02-fc9f-4c20-9f61-a2b60b10f3d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:38.125036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f2d02-fc9f-4c20-9f61-a2b60b10f3d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:38.125314 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f2d02-fc9f-4c20-9f61-a2b60b10f3d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:38.135288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f2d02-fc9f-4c20-9f61-a2b60b10f3d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:38.136751 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f2d02-fc9f-4c20-9f61-a2b60b10f3d5 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:38.146515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 693/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:39.478594 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eebbc-4df5-4acd-a5c2-d024d6bea9bc None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:39.479128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eebbc-4df5-4acd-a5c2-d024d6bea9bc None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:39.479255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eebbc-4df5-4acd-a5c2-d024d6bea9bc None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:39.480718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 684/1377] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:39.485926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9114e1c-4e33-4473-93ac-24628147b4c8 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:39.486398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9114e1c-4e33-4473-93ac-24628147b4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:39.486526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9114e1c-4e33-4473-93ac-24628147b4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:39.504744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c9114e1c-4e33-4473-93ac-24628147b4c8 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:39.505999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c9114e1c-4e33-4473-93ac-24628147b4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:39.513750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 694/1378] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:12:39.557591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813eebbc-4df5-4acd-a5c2-d024d6bea9bc None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:39.571661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813eebbc-4df5-4acd-a5c2-d024d6bea9bc None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:39.577044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996fd78d-4109-43b7-8266-b248a2203b16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:39.577682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996fd78d-4109-43b7-8266-b248a2203b16 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:39.577965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996fd78d-4109-43b7-8266-b248a2203b16 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:39.578240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996fd78d-4109-43b7-8266-b248a2203b16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:39.586770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996fd78d-4109-43b7-8266-b248a2203b16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:39.588446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996fd78d-4109-43b7-8266-b248a2203b16 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:39.597845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 685/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:41.776198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1fbc-712a-44de-880a-0f9f2c9796b2 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:41.776761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1fbc-712a-44de-880a-0f9f2c9796b2 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:41.777061 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1fbc-712a-44de-880a-0f9f2c9796b2 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:41.778685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 695/1380] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:41.783894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b9b84-20a7-4dee-9a37-3b39c14e0e31 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:41.784332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b9b84-20a7-4dee-9a37-3b39c14e0e31 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:41.784496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b9b84-20a7-4dee-9a37-3b39c14e0e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:41.802959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-f18b9b84-20a7-4dee-9a37-3b39c14e0e31 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:41.804194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f18b9b84-20a7-4dee-9a37-3b39c14e0e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:41.812300 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 686/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:41.858153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1fbc-712a-44de-880a-0f9f2c9796b2 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:41.872579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1fbc-712a-44de-880a-0f9f2c9796b2 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:41.878280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62eac6-841d-4d38-afe3-0b264736801d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:41.879092 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62eac6-841d-4d38-afe3-0b264736801d service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:41.879546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62eac6-841d-4d38-afe3-0b264736801d service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:41.879980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62eac6-841d-4d38-afe3-0b264736801d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:41.889265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62eac6-841d-4d38-afe3-0b264736801d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:41.891036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62eac6-841d-4d38-afe3-0b264736801d service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:41.901299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 696/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:44.070444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42246f-4da0-46b4-acb1-f57406eddd7e None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:44.070957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42246f-4da0-46b4-acb1-f57406eddd7e None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:44.071054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42246f-4da0-46b4-acb1-f57406eddd7e None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:44.072557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 687/1383] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:44.077619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c329e2-1861-452a-bc55-e04ace780c77 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:44.078137 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c329e2-1861-452a-bc55-e04ace780c77 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:44.078236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c329e2-1861-452a-bc55-e04ace780c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:44.097281 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-22c329e2-1861-452a-bc55-e04ace780c77 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:44.098469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-22c329e2-1861-452a-bc55-e04ace780c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:44.106626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 697/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:44.148365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42246f-4da0-46b4-acb1-f57406eddd7e None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:44.164187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42246f-4da0-46b4-acb1-f57406eddd7e None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:44.169802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c81678-7e1f-4e55-a197-b11b37c5dc42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:44.170487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c81678-7e1f-4e55-a197-b11b37c5dc42 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:44.170777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c81678-7e1f-4e55-a197-b11b37c5dc42 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:44.171058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c81678-7e1f-4e55-a197-b11b37c5dc42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:44.180604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c81678-7e1f-4e55-a197-b11b37c5dc42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:44.182203 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c81678-7e1f-4e55-a197-b11b37c5dc42 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:44.192195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 688/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:46.358059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828f372-d00e-4b93-85f0-5c7799b33192 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:46.358621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828f372-d00e-4b93-85f0-5c7799b33192 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:46.358914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828f372-d00e-4b93-85f0-5c7799b33192 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:46.360675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 698/1386] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:46.365978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349c224-2f89-4740-99f4-83bce82c1625 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:46.366535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349c224-2f89-4740-99f4-83bce82c1625 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:46.366535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349c224-2f89-4740-99f4-83bce82c1625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:46.385062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c349c224-2f89-4740-99f4-83bce82c1625 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:46.386372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c349c224-2f89-4740-99f4-83bce82c1625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:46.398589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 689/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:46.443942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828f372-d00e-4b93-85f0-5c7799b33192 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:46.461487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828f372-d00e-4b93-85f0-5c7799b33192 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:46.468127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f6bbd-0e56-43db-86a5-9b9c29ccddca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:46.468918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f6bbd-0e56-43db-86a5-9b9c29ccddca service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:46.469275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f6bbd-0e56-43db-86a5-9b9c29ccddca service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:46.469884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f6bbd-0e56-43db-86a5-9b9c29ccddca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:46.480530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f6bbd-0e56-43db-86a5-9b9c29ccddca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:46.482151 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f6bbd-0e56-43db-86a5-9b9c29ccddca service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:46.491991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 699/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:48.690128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f133c-5603-49f1-ac4e-1920f8bc5b18 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:48.690650 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f133c-5603-49f1-ac4e-1920f8bc5b18 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:48.690734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f133c-5603-49f1-ac4e-1920f8bc5b18 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:48.692100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 690/1389] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:48.698057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555fe5b-f0cf-479e-bb17-2a9a3dee046d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:48.698492 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555fe5b-f0cf-479e-bb17-2a9a3dee046d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:48.698652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555fe5b-f0cf-479e-bb17-2a9a3dee046d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:48.718002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-7555fe5b-f0cf-479e-bb17-2a9a3dee046d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:48.719258 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7555fe5b-f0cf-479e-bb17-2a9a3dee046d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:48.727384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 700/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:12:48.776646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f133c-5603-49f1-ac4e-1920f8bc5b18 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:48.794757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f133c-5603-49f1-ac4e-1920f8bc5b18 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:48.800630 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13cce1-9c34-4caa-977a-db58f6520ab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:48.801291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13cce1-9c34-4caa-977a-db58f6520ab7 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:48.801695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13cce1-9c34-4caa-977a-db58f6520ab7 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:48.801984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13cce1-9c34-4caa-977a-db58f6520ab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:48.810825 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13cce1-9c34-4caa-977a-db58f6520ab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:48.812482 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13cce1-9c34-4caa-977a-db58f6520ab7 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:48.822585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 691/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:50.994222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76994124-26da-4878-9258-1347df37614c None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:50.994804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76994124-26da-4878-9258-1347df37614c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:50.994804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76994124-26da-4878-9258-1347df37614c None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:50.996191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 701/1392] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:51.002675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fdc247-37f0-4cc3-883b-2f16d10c49e4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:51.003095 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fdc247-37f0-4cc3-883b-2f16d10c49e4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:51.003444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fdc247-37f0-4cc3-883b-2f16d10c49e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:51.025563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-f2fdc247-37f0-4cc3-883b-2f16d10c49e4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:51.027644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f2fdc247-37f0-4cc3-883b-2f16d10c49e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:51.036545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 692/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:51.083195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76994124-26da-4878-9258-1347df37614c None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:51.101435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76994124-26da-4878-9258-1347df37614c None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:51.107772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf974238-e5c0-4762-9a59-1c6747c59038 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:51.108546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf974238-e5c0-4762-9a59-1c6747c59038 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:51.108902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf974238-e5c0-4762-9a59-1c6747c59038 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:51.109235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf974238-e5c0-4762-9a59-1c6747c59038 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:51.119294 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf974238-e5c0-4762-9a59-1c6747c59038 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:51.121315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf974238-e5c0-4762-9a59-1c6747c59038 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:51.133523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 702/1394] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:53.255368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067af25-11d0-49a3-a043-02a1f1ca1acb None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:53.255913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067af25-11d0-49a3-a043-02a1f1ca1acb None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:53.255913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067af25-11d0-49a3-a043-02a1f1ca1acb None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:53.257994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 693/1395] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:53.263280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858597c-3654-419a-b006-acec9b3b9a1b None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:53.263735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858597c-3654-419a-b006-acec9b3b9a1b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:53.263905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858597c-3654-419a-b006-acec9b3b9a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:53.281832 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-8858597c-3654-419a-b006-acec9b3b9a1b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:53.283003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-8858597c-3654-419a-b006-acec9b3b9a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:53.291069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 703/1396] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:53.335986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067af25-11d0-49a3-a043-02a1f1ca1acb None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:53.350777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067af25-11d0-49a3-a043-02a1f1ca1acb None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:53.356302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b53776-19f2-4b3c-9a79-c1db8776b24e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:53.356991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b53776-19f2-4b3c-9a79-c1db8776b24e service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:53.357275 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b53776-19f2-4b3c-9a79-c1db8776b24e service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:53.357546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b53776-19f2-4b3c-9a79-c1db8776b24e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:53.366573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b53776-19f2-4b3c-9a79-c1db8776b24e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:53.368207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b53776-19f2-4b3c-9a79-c1db8776b24e service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:53.377854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 694/1397] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:53.400727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3110-97cc-417d-a7e5-84328e3f2ca1 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:53.401027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3110-97cc-417d-a7e5-84328e3f2ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:53.401313 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3110-97cc-417d-a7e5-84328e3f2ca1 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:53.402835 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 704/1398] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:53.412311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620440a-abca-41e0-bd82-6ddceb95b7af None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:53.412311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620440a-abca-41e0-bd82-6ddceb95b7af None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:53.412311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620440a-abca-41e0-bd82-6ddceb95b7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:53.426578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c620440a-abca-41e0-bd82-6ddceb95b7af None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:53.427773 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c620440a-abca-41e0-bd82-6ddceb95b7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:53.436051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 695/1399] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:53.495418 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de1172-d277-4040-a869-8790b5dfa14e None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:53.495711 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de1172-d277-4040-a869-8790b5dfa14e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:53.495997 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de1172-d277-4040-a869-8790b5dfa14e None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:53.497452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 705/1400] 10.222.0.33 () {54 vars in 962 bytes} [Fri Jun 17 07:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:53.503191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47700f30-fcd9-4026-b057-08395a2925ea None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:53.503483 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47700f30-fcd9-4026-b057-08395a2925ea None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:53.503759 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47700f30-fcd9-4026-b057-08395a2925ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:53.558807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-47700f30-fcd9-4026-b057-08395a2925ea None None] MFA Rules not processed for user `48586ff1269f49a3ae3d9f3e40c1b9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:53.560056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-47700f30-fcd9-4026-b057-08395a2925ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:53.596058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:12:53.625140 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 696/1401] 10.222.0.33 () {58 vars in 1059 bytes} [Fri Jun 17 07:12:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:53.633253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de1172-d277-4040-a869-8790b5dfa14e None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:53.643812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de1172-d277-4040-a869-8790b5dfa14e None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:53.647926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48feadd-94fd-4c13-ad0b-059d2ef1ae35 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48586ff1269f49a3ae3d9f3e40c1b9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:53.648604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48feadd-94fd-4c13-ad0b-059d2ef1ae35 service glance] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:53.648890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48feadd-94fd-4c13-ad0b-059d2ef1ae35 service glance] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:53.649172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48feadd-94fd-4c13-ad0b-059d2ef1ae35 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:53.659054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48feadd-94fd-4c13-ad0b-059d2ef1ae35 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:53.660521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48feadd-94fd-4c13-ad0b-059d2ef1ae35 service glance] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:53.670584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 706/1402] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:12:54.379830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b405e-8f0a-44b2-b825-7a1f73f7b7d4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:54.380344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b405e-8f0a-44b2-b825-7a1f73f7b7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:54.380451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b405e-8f0a-44b2-b825-7a1f73f7b7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:54.429133 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-253b405e-8f0a-44b2-b825-7a1f73f7b7d4 None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:54.430341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-253b405e-8f0a-44b2-b825-7a1f73f7b7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:54.493224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 697/1403] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:54.536423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadebda5-6887-4867-be6f-582112cf09fc None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:54.536954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadebda5-6887-4867-be6f-582112cf09fc None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:54.537188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadebda5-6887-4867-be6f-582112cf09fc None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:54.538611 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 707/1404] 10.222.0.33 () {54 vars in 942 bytes} [Fri Jun 17 07:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:54.544921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e56a5b-e87b-47f2-8283-08fd85321012 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:54.545274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e56a5b-e87b-47f2-8283-08fd85321012 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:54.545618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e56a5b-e87b-47f2-8283-08fd85321012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:54.593177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-76e56a5b-e87b-47f2-8283-08fd85321012 None None] MFA Rules not processed for user `004e5909f9834cfea890063bb0083d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:54.594364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-76e56a5b-e87b-47f2-8283-08fd85321012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:54.664454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 698/1405] 10.222.0.33 () {58 vars in 1039 bytes} [Fri Jun 17 07:12:54 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:54.673837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadebda5-6887-4867-be6f-582112cf09fc None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:54.687719 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadebda5-6887-4867-be6f-582112cf09fc None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:54.692705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c314487-0a4c-433f-9549-38f8b52e21de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:54.693344 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c314487-0a4c-433f-9549-38f8b52e21de service swift] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:54.693665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c314487-0a4c-433f-9549-38f8b52e21de service swift] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:54.693948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c314487-0a4c-433f-9549-38f8b52e21de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:54.701889 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c314487-0a4c-433f-9549-38f8b52e21de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:54.703472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c314487-0a4c-433f-9549-38f8b52e21de service swift] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:54.711428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 708/1406] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:12:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:12:55.546047 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141f44c-c526-402f-a860-2abc209a9f14 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:55.546587 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141f44c-c526-402f-a860-2abc209a9f14 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:55.546587 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141f44c-c526-402f-a860-2abc209a9f14 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:55.547972 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 699/1407] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:12:55.553533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9418bb8-7c21-4c7d-ab19-136515aaa6dd None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:55.553952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9418bb8-7c21-4c7d-ab19-136515aaa6dd None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:55.554093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9418bb8-7c21-4c7d-ab19-136515aaa6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:55.560365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=67 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:12:55.573894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-a9418bb8-7c21-4c7d-ab19-136515aaa6dd None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:12:55.575096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a9418bb8-7c21-4c7d-ab19-136515aaa6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:12:55.583080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 709/1408] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:12:55.626847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141f44c-c526-402f-a860-2abc209a9f14 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:12:55.645362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141f44c-c526-402f-a860-2abc209a9f14 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:12:55.650797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7ea5c-dd72-4762-886a-58f992b43d3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:12:55.651583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7ea5c-dd72-4762-886a-58f992b43d3a service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:12:55.651872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7ea5c-dd72-4762-886a-58f992b43d3a service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:12:55.652187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7ea5c-dd72-4762-886a-58f992b43d3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:12:55.661095 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7ea5c-dd72-4762-886a-58f992b43d3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:12:55.663349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7ea5c-dd72-4762-886a-58f992b43d3a service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:12:55.668908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:12:55.674893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 700/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:12:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:05.568488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:13:17.079515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc881d4-f5df-44cc-bf93-7a1f5710f581 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:17.080206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc881d4-f5df-44cc-bf93-7a1f5710f581 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:17.080206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc881d4-f5df-44cc-bf93-7a1f5710f581 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:17.081514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 710/1410] 10.222.0.33 () {54 vars in 918 bytes} [Fri Jun 17 07:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:17.086556 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87382db-aecf-4c9f-9a44-3d3d5edf90af None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:17.086980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87382db-aecf-4c9f-9a44-3d3d5edf90af None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:17.087102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87382db-aecf-4c9f-9a44-3d3d5edf90af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:17.105771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-e87382db-aecf-4c9f-9a44-3d3d5edf90af None None] MFA Rules not processed for user `b8cf02a52d354517b363c0aaf477550c`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:17.106953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e87382db-aecf-4c9f-9a44-3d3d5edf90af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:17.115429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 701/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Fri Jun 17 07:13:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:17.129532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3081115-b735-4e31-a05d-abd2a92341ff None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:17.129827 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3081115-b735-4e31-a05d-abd2a92341ff None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:17.130109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3081115-b735-4e31-a05d-abd2a92341ff None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:17.132019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 711/1412] 10.222.0.33 () {54 vars in 918 bytes} [Fri Jun 17 07:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:17.137266 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ad1bf-146b-4ecf-9b0c-5a32ee91700d None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:17.137674 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ad1bf-146b-4ecf-9b0c-5a32ee91700d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:17.137958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ad1bf-146b-4ecf-9b0c-5a32ee91700d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:17.156034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c28ad1bf-146b-4ecf-9b0c-5a32ee91700d None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:17.157253 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c28ad1bf-146b-4ecf-9b0c-5a32ee91700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:17.167078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 702/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Fri Jun 17 07:13:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:13:19.733472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a36442-b8be-416d-8df6-24e7a0a91ca6 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:19.734068 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a36442-b8be-416d-8df6-24e7a0a91ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:19.734068 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a36442-b8be-416d-8df6-24e7a0a91ca6 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:19.735373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:19.740527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358ab00-a543-4831-afb7-56f169e5ee3d None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:19.741496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358ab00-a543-4831-afb7-56f169e5ee3d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:19.741636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358ab00-a543-4831-afb7-56f169e5ee3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:19.758993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-5358ab00-a543-4831-afb7-56f169e5ee3d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:19.760224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5358ab00-a543-4831-afb7-56f169e5ee3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:19.767915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:13:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:19.810468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a36442-b8be-416d-8df6-24e7a0a91ca6 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:19.825192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a36442-b8be-416d-8df6-24e7a0a91ca6 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:19.830541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5905667-e590-4a15-8c01-d7034995fd90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:19.831177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5905667-e590-4a15-8c01-d7034995fd90 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:19.831465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5905667-e590-4a15-8c01-d7034995fd90 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:19.831741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5905667-e590-4a15-8c01-d7034995fd90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:19.840275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5905667-e590-4a15-8c01-d7034995fd90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:19.841846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5905667-e590-4a15-8c01-d7034995fd90 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:19.851691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 713/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:20.793074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419877b-bf38-493a-b496-70db76f55976 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:20.793653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419877b-bf38-493a-b496-70db76f55976 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:20.793653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419877b-bf38-493a-b496-70db76f55976 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:20.795039 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 07:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:20.799772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edce2b6-f5a4-419e-abd2-d3ae7d923021 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:20.800215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edce2b6-f5a4-419e-abd2-d3ae7d923021 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:20.800358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edce2b6-f5a4-419e-abd2-d3ae7d923021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:20.818235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-0edce2b6-f5a4-419e-abd2-d3ae7d923021 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:20.819425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-0edce2b6-f5a4-419e-abd2-d3ae7d923021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:20.827100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 07:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:20.978422 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419877b-bf38-493a-b496-70db76f55976 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:20.993430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419877b-bf38-493a-b496-70db76f55976 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:20.998905 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84dbbb-739f-474d-a04f-bddc58213b8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:20.999522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84dbbb-739f-474d-a04f-bddc58213b8a service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:20.999806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84dbbb-739f-474d-a04f-bddc58213b8a service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:21.000110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84dbbb-739f-474d-a04f-bddc58213b8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:21.008076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84dbbb-739f-474d-a04f-bddc58213b8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:21.009671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84dbbb-739f-474d-a04f-bddc58213b8a service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:21.018884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 705/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:21.943184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16240277-ceeb-42bd-9ec7-c294c3ffa0e1 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:21.943759 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16240277-ceeb-42bd-9ec7-c294c3ffa0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:21.943759 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16240277-ceeb-42bd-9ec7-c294c3ffa0e1 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:21.945183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 07:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:21.949942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604309d9-8af7-42bf-b4bd-5114029f38ba None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:21.950370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604309d9-8af7-42bf-b4bd-5114029f38ba None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:21.950534 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604309d9-8af7-42bf-b4bd-5114029f38ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:21.968178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-604309d9-8af7-42bf-b4bd-5114029f38ba None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:21.969477 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-604309d9-8af7-42bf-b4bd-5114029f38ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:21.977755 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 07:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:22.128455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16240277-ceeb-42bd-9ec7-c294c3ffa0e1 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:22.142961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16240277-ceeb-42bd-9ec7-c294c3ffa0e1 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:22.148837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9db73-ad36-4862-979f-8a4d96a4e1a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:22.149476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9db73-ad36-4862-979f-8a4d96a4e1a5 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:22.149764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9db73-ad36-4862-979f-8a4d96a4e1a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:22.150044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9db73-ad36-4862-979f-8a4d96a4e1a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:22.158035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9db73-ad36-4862-979f-8a4d96a4e1a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:22.159611 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9db73-ad36-4862-979f-8a4d96a4e1a5 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:22.169660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 716/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:23.064497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc93c9e-02dc-4950-b3dd-c1b396546acc None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:23.065061 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc93c9e-02dc-4950-b3dd-c1b396546acc None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:23.065061 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc93c9e-02dc-4950-b3dd-c1b396546acc None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:23.066441 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 07:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:23.071586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06522f0c-2402-4309-a7ff-5c6445f0ef98 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:23.071987 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06522f0c-2402-4309-a7ff-5c6445f0ef98 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:23.072193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06522f0c-2402-4309-a7ff-5c6445f0ef98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:23.090867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-06522f0c-2402-4309-a7ff-5c6445f0ef98 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:23.092041 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-06522f0c-2402-4309-a7ff-5c6445f0ef98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:23.099735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 07:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:23.257852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc93c9e-02dc-4950-b3dd-c1b396546acc None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:23.272050 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc93c9e-02dc-4950-b3dd-c1b396546acc None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:23.277978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557469c-ba2f-4d1f-a3e3-45dde143f5d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:23.278613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557469c-ba2f-4d1f-a3e3-45dde143f5d6 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:23.278902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557469c-ba2f-4d1f-a3e3-45dde143f5d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:23.279184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557469c-ba2f-4d1f-a3e3-45dde143f5d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:23.287399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557469c-ba2f-4d1f-a3e3-45dde143f5d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:23.289064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557469c-ba2f-4d1f-a3e3-45dde143f5d6 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:23.298347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 708/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:24.195025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee138c-209e-4bc1-8ca1-bb1bf0ac5fb5 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:24.195615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee138c-209e-4bc1-8ca1-bb1bf0ac5fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:24.195615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee138c-209e-4bc1-8ca1-bb1bf0ac5fb5 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:24.196973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 07:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:24.201777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4051f1e-92a1-4105-a70e-45c68447557d None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:24.202206 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4051f1e-92a1-4105-a70e-45c68447557d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:24.202318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4051f1e-92a1-4105-a70e-45c68447557d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:24.220537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-b4051f1e-92a1-4105-a70e-45c68447557d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:24.221744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b4051f1e-92a1-4105-a70e-45c68447557d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:24.229744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 07:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:24.379338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee138c-209e-4bc1-8ca1-bb1bf0ac5fb5 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:24.394592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee138c-209e-4bc1-8ca1-bb1bf0ac5fb5 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:24.399958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388ef71-01ed-4a0e-bb5b-1eff1659f14a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:24.400621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388ef71-01ed-4a0e-bb5b-1eff1659f14a service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:24.400904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388ef71-01ed-4a0e-bb5b-1eff1659f14a service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:24.401178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388ef71-01ed-4a0e-bb5b-1eff1659f14a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:24.409067 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388ef71-01ed-4a0e-bb5b-1eff1659f14a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:24.410881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388ef71-01ed-4a0e-bb5b-1eff1659f14a service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:24.419982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 719/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:25.258466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb6f5a-9a1c-4dec-be1d-bce68093b034 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:25.259105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb6f5a-9a1c-4dec-be1d-bce68093b034 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:25.259105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb6f5a-9a1c-4dec-be1d-bce68093b034 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:25.260419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 07:13:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:25.265329 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61668a-6b47-4678-a2dd-09d16ef8ddf9 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:25.265752 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61668a-6b47-4678-a2dd-09d16ef8ddf9 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:25.265894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61668a-6b47-4678-a2dd-09d16ef8ddf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:25.283985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3c61668a-6b47-4678-a2dd-09d16ef8ddf9 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:25.285467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3c61668a-6b47-4678-a2dd-09d16ef8ddf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:25.293808 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 07:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:25.445094 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb6f5a-9a1c-4dec-be1d-bce68093b034 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:25.461256 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb6f5a-9a1c-4dec-be1d-bce68093b034 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:25.466614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28523196-7a82-434c-a604-3168893afa42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:25.467248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28523196-7a82-434c-a604-3168893afa42 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:25.467534 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28523196-7a82-434c-a604-3168893afa42 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:25.467815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28523196-7a82-434c-a604-3168893afa42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:25.475715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28523196-7a82-434c-a604-3168893afa42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:25.477329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28523196-7a82-434c-a604-3168893afa42 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:25.486742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 711/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:26.326509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aba738-246b-46cf-a968-2f68d0d63506 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:26.327084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aba738-246b-46cf-a968-2f68d0d63506 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:26.327084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aba738-246b-46cf-a968-2f68d0d63506 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:26.328474 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 07:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:13:26.333317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051fe39-6413-4f3a-83b7-936fcf3c1bb3 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:26.333759 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051fe39-6413-4f3a-83b7-936fcf3c1bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:26.333865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051fe39-6413-4f3a-83b7-936fcf3c1bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:26.352840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-0051fe39-6413-4f3a-83b7-936fcf3c1bb3 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:26.354017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-0051fe39-6413-4f3a-83b7-936fcf3c1bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:26.361715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 07:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:26.511981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aba738-246b-46cf-a968-2f68d0d63506 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:26.526373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aba738-246b-46cf-a968-2f68d0d63506 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:26.531825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2b02f-3d43-4c9d-9a92-4f77b9a3c731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:26.532494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2b02f-3d43-4c9d-9a92-4f77b9a3c731 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:26.532778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2b02f-3d43-4c9d-9a92-4f77b9a3c731 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:26.533053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2b02f-3d43-4c9d-9a92-4f77b9a3c731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:26.542406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2b02f-3d43-4c9d-9a92-4f77b9a3c731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:26.543965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2b02f-3d43-4c9d-9a92-4f77b9a3c731 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:26.553812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 722/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:27.394323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e697a0-b21f-47dc-ad14-62a2e7f78038 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:27.394758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e697a0-b21f-47dc-ad14-62a2e7f78038 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:27.394863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e697a0-b21f-47dc-ad14-62a2e7f78038 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:27.396273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 07:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:27.401126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdf237-d9d2-4a5b-88b7-d5935848e987 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:27.401574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdf237-d9d2-4a5b-88b7-d5935848e987 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:27.401681 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdf237-d9d2-4a5b-88b7-d5935848e987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:27.419469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-1cbdf237-d9d2-4a5b-88b7-d5935848e987 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:27.420736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-1cbdf237-d9d2-4a5b-88b7-d5935848e987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:27.429069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 07:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:27.586589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e697a0-b21f-47dc-ad14-62a2e7f78038 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:27.600675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e697a0-b21f-47dc-ad14-62a2e7f78038 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:27.606023 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a148e-e319-4b36-8cec-902c8e9f160a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:27.606651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a148e-e319-4b36-8cec-902c8e9f160a service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:27.606933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a148e-e319-4b36-8cec-902c8e9f160a service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:27.607209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a148e-e319-4b36-8cec-902c8e9f160a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:27.615146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a148e-e319-4b36-8cec-902c8e9f160a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:27.617276 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a148e-e319-4b36-8cec-902c8e9f160a service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:27.626633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 714/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:28.475224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea7af-aba6-45a5-a420-d5c5cce003a3 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:28.475775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea7af-aba6-45a5-a420-d5c5cce003a3 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:28.475775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aea7af-aba6-45a5-a420-d5c5cce003a3 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:28.477183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 07:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:28.482066 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb082-e385-43f3-aeb8-a27890b728dd None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:28.482494 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb082-e385-43f3-aeb8-a27890b728dd None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:28.482609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb082-e385-43f3-aeb8-a27890b728dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:28.499939 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c82fb082-e385-43f3-aeb8-a27890b728dd None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:28.501144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c82fb082-e385-43f3-aeb8-a27890b728dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:28.509334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 07:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:28.659783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aea7af-aba6-45a5-a420-d5c5cce003a3 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:28.674169 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aea7af-aba6-45a5-a420-d5c5cce003a3 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:28.679494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ab514-cb50-404c-9578-39155f8571d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:28.680156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ab514-cb50-404c-9578-39155f8571d3 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:28.680445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ab514-cb50-404c-9578-39155f8571d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:28.680722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ab514-cb50-404c-9578-39155f8571d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:28.688618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ab514-cb50-404c-9578-39155f8571d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:28.690190 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ab514-cb50-404c-9578-39155f8571d3 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:28.699878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 725/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:30.704388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8717a-6256-49a6-8974-131f2b9a43ae None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:30.704934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8717a-6256-49a6-8974-131f2b9a43ae None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:30.704934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8717a-6256-49a6-8974-131f2b9a43ae None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:30.706325 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:30.712193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196aa8b-38ad-4387-b2b3-cf1c6ef4b979 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:30.712576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196aa8b-38ad-4387-b2b3-cf1c6ef4b979 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:30.712757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196aa8b-38ad-4387-b2b3-cf1c6ef4b979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:30.730013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3196aa8b-38ad-4387-b2b3-cf1c6ef4b979 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:30.731199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3196aa8b-38ad-4387-b2b3-cf1c6ef4b979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:30.738857 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:30.777835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8717a-6256-49a6-8974-131f2b9a43ae None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:30.788598 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8717a-6256-49a6-8974-131f2b9a43ae None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:30.792092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1523c1-941e-43ba-becc-d4926c44c573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:30.792739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1523c1-941e-43ba-becc-d4926c44c573 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:30.793029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1523c1-941e-43ba-becc-d4926c44c573 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:30.793737 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1523c1-941e-43ba-becc-d4926c44c573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:30.802152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1523c1-941e-43ba-becc-d4926c44c573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:30.803590 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1523c1-941e-43ba-becc-d4926c44c573 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:30.812828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 717/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:33.075635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfae6d-2421-4e49-8263-e5c618e5bfbc None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:33.076220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfae6d-2421-4e49-8263-e5c618e5bfbc None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:33.076220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfae6d-2421-4e49-8263-e5c618e5bfbc None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:33.077649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:33.082995 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9e599-44ac-4297-b177-4a5a85e21b00 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:33.083404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9e599-44ac-4297-b177-4a5a85e21b00 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:33.083595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9e599-44ac-4297-b177-4a5a85e21b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:33.101395 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-48c9e599-44ac-4297-b177-4a5a85e21b00 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:33.102604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-48c9e599-44ac-4297-b177-4a5a85e21b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:33.110515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:33.148907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfae6d-2421-4e49-8263-e5c618e5bfbc None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:33.159484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfae6d-2421-4e49-8263-e5c618e5bfbc None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:33.163712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901613b8-9d71-418d-b5d0-e9f658ac0815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:33.164379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901613b8-9d71-418d-b5d0-e9f658ac0815 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:33.164663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901613b8-9d71-418d-b5d0-e9f658ac0815 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:33.164944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901613b8-9d71-418d-b5d0-e9f658ac0815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:33.173772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901613b8-9d71-418d-b5d0-e9f658ac0815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:33.175185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901613b8-9d71-418d-b5d0-e9f658ac0815 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:33.185045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 728/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:35.573483 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8529a4c-d1fe-47d7-80c4-a5831d2974e3 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:35.573967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8529a4c-d1fe-47d7-80c4-a5831d2974e3 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:35.574064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8529a4c-d1fe-47d7-80c4-a5831d2974e3 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:35.575416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:35.580583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3656a86-12f1-4201-b687-b7accca2f6bd None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:35.580993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3656a86-12f1-4201-b687-b7accca2f6bd None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:35.581137 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3656a86-12f1-4201-b687-b7accca2f6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:35.598337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-e3656a86-12f1-4201-b687-b7accca2f6bd None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:35.599511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e3656a86-12f1-4201-b687-b7accca2f6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:35.607052 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:35.645824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8529a4c-d1fe-47d7-80c4-a5831d2974e3 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:35.658017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8529a4c-d1fe-47d7-80c4-a5831d2974e3 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:35.661871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77aa73-ccf2-488d-8700-9a40b9d99e7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:35.662513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77aa73-ccf2-488d-8700-9a40b9d99e7a service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:35.662830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77aa73-ccf2-488d-8700-9a40b9d99e7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:35.663647 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77aa73-ccf2-488d-8700-9a40b9d99e7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:35.672869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77aa73-ccf2-488d-8700-9a40b9d99e7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:35.674298 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77aa73-ccf2-488d-8700-9a40b9d99e7a service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:35.683593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 720/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:37.973864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39605b-83a3-488e-8b60-6688713c0858 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:37.974425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39605b-83a3-488e-8b60-6688713c0858 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:37.974425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39605b-83a3-488e-8b60-6688713c0858 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:37.976472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:37.981563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a6aef-c94b-4bc0-ad5f-141c904a557b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:37.981973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a6aef-c94b-4bc0-ad5f-141c904a557b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:37.982118 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a6aef-c94b-4bc0-ad5f-141c904a557b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:37.999256 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-1f9a6aef-c94b-4bc0-ad5f-141c904a557b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:38.000472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1f9a6aef-c94b-4bc0-ad5f-141c904a557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:38.008117 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:38.045949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39605b-83a3-488e-8b60-6688713c0858 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:38.056521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39605b-83a3-488e-8b60-6688713c0858 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:38.060309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a95913-5a25-460a-b7ef-755df9771e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:38.060888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a95913-5a25-460a-b7ef-755df9771e94 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:38.061171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a95913-5a25-460a-b7ef-755df9771e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:38.061445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a95913-5a25-460a-b7ef-755df9771e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:38.070316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a95913-5a25-460a-b7ef-755df9771e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:38.071733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a95913-5a25-460a-b7ef-755df9771e94 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:38.080881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 731/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:40.229613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0295a-5d9c-41eb-a5d3-e260a532c616 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:40.230185 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0295a-5d9c-41eb-a5d3-e260a532c616 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:40.230185 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0295a-5d9c-41eb-a5d3-e260a532c616 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:40.231523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:40.236773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96739f-78a4-461c-9375-312b8ae79195 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:40.237158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96739f-78a4-461c-9375-312b8ae79195 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:40.237366 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96739f-78a4-461c-9375-312b8ae79195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:40.254515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-8d96739f-78a4-461c-9375-312b8ae79195 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:40.255704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-8d96739f-78a4-461c-9375-312b8ae79195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:40.264252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:40.301957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0295a-5d9c-41eb-a5d3-e260a532c616 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:40.312793 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0295a-5d9c-41eb-a5d3-e260a532c616 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:40.316471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c720a-8adf-4b49-b720-678c916a4a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:40.317128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c720a-8adf-4b49-b720-678c916a4a60 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:40.317408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c720a-8adf-4b49-b720-678c916a4a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:40.317682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c720a-8adf-4b49-b720-678c916a4a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:40.327583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c720a-8adf-4b49-b720-678c916a4a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:40.329067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c720a-8adf-4b49-b720-678c916a4a60 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:40.340143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 723/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:42.500127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ae795-c11e-425a-961e-26502272cb1a None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:42.500657 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ae795-c11e-425a-961e-26502272cb1a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:42.500757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ae795-c11e-425a-961e-26502272cb1a None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:42.502055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:42.507214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bef9ff-f95f-411d-8e49-cd3fefe2c144 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:42.507621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bef9ff-f95f-411d-8e49-cd3fefe2c144 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:42.507777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bef9ff-f95f-411d-8e49-cd3fefe2c144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:42.525439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-56bef9ff-f95f-411d-8e49-cd3fefe2c144 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:42.526629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-56bef9ff-f95f-411d-8e49-cd3fefe2c144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:42.534353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:42.571545 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ae795-c11e-425a-961e-26502272cb1a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:42.582066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ae795-c11e-425a-961e-26502272cb1a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:42.586079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cf618-371b-45d5-b294-7cd5de172ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:42.586704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cf618-371b-45d5-b294-7cd5de172ecb service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:42.586989 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cf618-371b-45d5-b294-7cd5de172ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:42.587263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cf618-371b-45d5-b294-7cd5de172ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:42.595885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cf618-371b-45d5-b294-7cd5de172ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:42.597332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cf618-371b-45d5-b294-7cd5de172ecb service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:42.607003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 734/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:44.750873 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33d90e-52e3-4040-b138-e3bd4fec7107 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:44.751456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33d90e-52e3-4040-b138-e3bd4fec7107 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:44.751568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33d90e-52e3-4040-b138-e3bd4fec7107 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:44.752994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:44.758235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac940935-2fad-4a18-96da-5d0660413634 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:44.758671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac940935-2fad-4a18-96da-5d0660413634 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:44.758798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac940935-2fad-4a18-96da-5d0660413634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:44.776557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ac940935-2fad-4a18-96da-5d0660413634 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:44.777748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ac940935-2fad-4a18-96da-5d0660413634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:44.785345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:44.822361 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33d90e-52e3-4040-b138-e3bd4fec7107 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:44.833000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33d90e-52e3-4040-b138-e3bd4fec7107 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:44.837086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41b608-6f73-4b00-b1a9-b6da9bf6e0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:44.837719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41b608-6f73-4b00-b1a9-b6da9bf6e0ce service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:44.838007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41b608-6f73-4b00-b1a9-b6da9bf6e0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:44.838287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41b608-6f73-4b00-b1a9-b6da9bf6e0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:44.847179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41b608-6f73-4b00-b1a9-b6da9bf6e0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:44.848655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41b608-6f73-4b00-b1a9-b6da9bf6e0ce service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:44.858646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 726/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:46.994642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3cfa0-5c01-4404-a4bb-18872edeecf8 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:46.995182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3cfa0-5c01-4404-a4bb-18872edeecf8 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:46.995182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3cfa0-5c01-4404-a4bb-18872edeecf8 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:46.996636 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:47.001814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc48d19-c994-4b43-a41f-347d8836d7fc None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:47.002246 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc48d19-c994-4b43-a41f-347d8836d7fc None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:47.002385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc48d19-c994-4b43-a41f-347d8836d7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:47.009048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=79 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:13:47.021316 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-fdc48d19-c994-4b43-a41f-347d8836d7fc None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:47.022526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-fdc48d19-c994-4b43-a41f-347d8836d7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:47.030263 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:47.066903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3cfa0-5c01-4404-a4bb-18872edeecf8 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:47.072901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=73 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:13:47.079207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3cfa0-5c01-4404-a4bb-18872edeecf8 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:47.082742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00be3f-08af-4bba-8715-1a8d42f52146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:47.083375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00be3f-08af-4bba-8715-1a8d42f52146 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:47.083668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00be3f-08af-4bba-8715-1a8d42f52146 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:47.083949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00be3f-08af-4bba-8715-1a8d42f52146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:47.092816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00be3f-08af-4bba-8715-1a8d42f52146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:47.094244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00be3f-08af-4bba-8715-1a8d42f52146 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:47.103441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 737/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:49.235789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdd1b0-aa5f-44ab-b9dd-37c47a2a147f None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:49.236416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdd1b0-aa5f-44ab-b9dd-37c47a2a147f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:49.236416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdd1b0-aa5f-44ab-b9dd-37c47a2a147f None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:49.237757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:49.243057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b9acc-a140-4ef3-928f-f6ab59e41eaf None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:49.243469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b9acc-a140-4ef3-928f-f6ab59e41eaf None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:49.243625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b9acc-a140-4ef3-928f-f6ab59e41eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:49.260923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-a26b9acc-a140-4ef3-928f-f6ab59e41eaf None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:49.262119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a26b9acc-a140-4ef3-928f-f6ab59e41eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:49.269785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:49.306126 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdd1b0-aa5f-44ab-b9dd-37c47a2a147f None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:49.316848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdd1b0-aa5f-44ab-b9dd-37c47a2a147f None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:49.320983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c16add-83a1-4fdd-810e-41ca0c1c22f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:49.321617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c16add-83a1-4fdd-810e-41ca0c1c22f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:49.321898 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c16add-83a1-4fdd-810e-41ca0c1c22f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:49.322173 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c16add-83a1-4fdd-810e-41ca0c1c22f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:49.330915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c16add-83a1-4fdd-810e-41ca0c1c22f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:49.332373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c16add-83a1-4fdd-810e-41ca0c1c22f3 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:49.342060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 729/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:13:51.451668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95891bf8-262c-4ac0-bcad-eb12fcc58a0e None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:51.452278 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95891bf8-262c-4ac0-bcad-eb12fcc58a0e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:51.452278 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95891bf8-262c-4ac0-bcad-eb12fcc58a0e None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:51.453628 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:13:51.458934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae8637-a168-4dcc-8efd-08ce35a4ee67 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:51.459389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae8637-a168-4dcc-8efd-08ce35a4ee67 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:51.459532 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae8637-a168-4dcc-8efd-08ce35a4ee67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:51.477970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-fdae8637-a168-4dcc-8efd-08ce35a4ee67 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:13:51.479183 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-fdae8637-a168-4dcc-8efd-08ce35a4ee67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:13:51.487045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:13:51.523050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95891bf8-262c-4ac0-bcad-eb12fcc58a0e None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:13:51.533708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95891bf8-262c-4ac0-bcad-eb12fcc58a0e None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:13:51.537518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6ce42-83f0-4001-834e-12c9e2f077ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:13:51.538710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6ce42-83f0-4001-834e-12c9e2f077ef service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:13:51.538992 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6ce42-83f0-4001-834e-12c9e2f077ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:13:51.539276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6ce42-83f0-4001-834e-12c9e2f077ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:13:51.548024 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6ce42-83f0-4001-834e-12c9e2f077ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:13:51.549477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6ce42-83f0-4001-834e-12c9e2f077ef service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:13:51.558790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 740/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:14:01.472331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=11 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:14:01.554945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=14 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:14:11.344898 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629966b-6983-4201-b2d5-e99348e91dea None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:11.345444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629966b-6983-4201-b2d5-e99348e91dea None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:11.345444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629966b-6983-4201-b2d5-e99348e91dea None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:11.346846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 731/1471] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:14:11.352273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdb5bc-656c-4169-bd48-d5d33a15c4a0 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:11.352749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdb5bc-656c-4169-bd48-d5d33a15c4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:11.352840 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdb5bc-656c-4169-bd48-d5d33a15c4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:11.370499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c0bdb5bc-656c-4169-bd48-d5d33a15c4a0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:14:11.371708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c0bdb5bc-656c-4169-bd48-d5d33a15c4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:14:11.379772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 741/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:14:11.427916 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f17a-3c91-4477-9d03-28caefbd5e42 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:11.428246 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f17a-3c91-4477-9d03-28caefbd5e42 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:11.428525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f17a-3c91-4477-9d03-28caefbd5e42 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:11.430762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 732/1473] 10.222.0.33 () {54 vars in 962 bytes} [Fri Jun 17 07:14:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:14:11.435510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a178278-93e7-467b-b68a-7f1e1ab6f5dd None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:11.435803 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a178278-93e7-467b-b68a-7f1e1ab6f5dd None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:11.436114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a178278-93e7-467b-b68a-7f1e1ab6f5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:11.454168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-8a178278-93e7-467b-b68a-7f1e1ab6f5dd None None] MFA Rules not processed for user `48586ff1269f49a3ae3d9f3e40c1b9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:14:11.455373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-8a178278-93e7-467b-b68a-7f1e1ab6f5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:14:11.462232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 742/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Fri Jun 17 07:14:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:14:11.469634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975f17a-3c91-4477-9d03-28caefbd5e42 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:14:11.482330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975f17a-3c91-4477-9d03-28caefbd5e42 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:14:11.485602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb1bc7-0af4-4bf4-b174-c1a14a98c7ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48586ff1269f49a3ae3d9f3e40c1b9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:14:11.486263 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb1bc7-0af4-4bf4-b174-c1a14a98c7ae service glance] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:11.486548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb1bc7-0af4-4bf4-b174-c1a14a98c7ae service glance] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:11.486829 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb1bc7-0af4-4bf4-b174-c1a14a98c7ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:11.497079 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb1bc7-0af4-4bf4-b174-c1a14a98c7ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:14:11.498857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb1bc7-0af4-4bf4-b174-c1a14a98c7ae service glance] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:14:11.508797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 733/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:14:13.681613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6814f-823c-42b5-b739-e40d37387575 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:13.682175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6814f-823c-42b5-b739-e40d37387575 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:13.682175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6814f-823c-42b5-b739-e40d37387575 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:13.683531 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 743/1476] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:14:13.688633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be800-ece2-40eb-80a2-f921a58e2a42 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:13.689102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be800-ece2-40eb-80a2-f921a58e2a42 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:13.689196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be800-ece2-40eb-80a2-f921a58e2a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:13.706344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-884be800-ece2-40eb-80a2-f921a58e2a42 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:14:13.707517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-884be800-ece2-40eb-80a2-f921a58e2a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:14:13.715266 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 734/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:14:13.737678 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6814f-823c-42b5-b739-e40d37387575 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:14:13.750183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6814f-823c-42b5-b739-e40d37387575 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:14:13.754472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6ee45-a101-4f99-8ad2-6dee6fb1fbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:14:13.755098 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6ee45-a101-4f99-8ad2-6dee6fb1fbda admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:13.755378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6ee45-a101-4f99-8ad2-6dee6fb1fbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:13.755652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6ee45-a101-4f99-8ad2-6dee6fb1fbda admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:13.756403 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6ee45-a101-4f99-8ad2-6dee6fb1fbda admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:14:13.758179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: WARNING py.warnings [None req-50b6ee45-a101-4f99-8ad2-6dee6fb1fbda admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:14:13.758179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: warnings.warn(msg) Jun 17 07:14:13.758179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: Jun 17 07:14:13.759308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6ee45-a101-4f99-8ad2-6dee6fb1fbda admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:14:13.778322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 744/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jun 17 07:14:13 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:14:15.919950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a46ee-8788-4d44-b06b-2912f6fd6f92 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:15.920546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a46ee-8788-4d44-b06b-2912f6fd6f92 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:15.920546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a46ee-8788-4d44-b06b-2912f6fd6f92 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:15.921922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 735/1479] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:14:15.927200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfb5e7-c9c1-4260-8790-47d297581458 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:15.927612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfb5e7-c9c1-4260-8790-47d297581458 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:15.927767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfb5e7-c9c1-4260-8790-47d297581458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:15.946556 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-acbfb5e7-c9c1-4260-8790-47d297581458 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:14:15.948209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-acbfb5e7-c9c1-4260-8790-47d297581458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:14:15.956950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 745/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:14:15.999661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a46ee-8788-4d44-b06b-2912f6fd6f92 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:14:16.013913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a46ee-8788-4d44-b06b-2912f6fd6f92 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:14:16.019326 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5494594-5f3c-4d27-8fdd-19149a2b32fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:14:16.019959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5494594-5f3c-4d27-8fdd-19149a2b32fe service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:16.020271 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5494594-5f3c-4d27-8fdd-19149a2b32fe service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:16.020552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5494594-5f3c-4d27-8fdd-19149a2b32fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:16.029047 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5494594-5f3c-4d27-8fdd-19149a2b32fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:14:16.030630 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5494594-5f3c-4d27-8fdd-19149a2b32fe service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:14:16.040699 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 736/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:14:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:14:18.314693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc051e5-71a8-4263-87fe-a516c1979045 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:18.315258 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc051e5-71a8-4263-87fe-a516c1979045 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:18.315258 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc051e5-71a8-4263-87fe-a516c1979045 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:18.316668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 746/1482] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:14:18.321758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c95c7-ae83-4286-8cd5-2976e3b41fba None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:18.322195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c95c7-ae83-4286-8cd5-2976e3b41fba None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:18.322335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c95c7-ae83-4286-8cd5-2976e3b41fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:18.340264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-be2c95c7-ae83-4286-8cd5-2976e3b41fba None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:14:18.341461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-be2c95c7-ae83-4286-8cd5-2976e3b41fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:14:18.349223 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 737/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:14:18.385245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc051e5-71a8-4263-87fe-a516c1979045 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:14:18.396236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc051e5-71a8-4263-87fe-a516c1979045 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:14:18.400245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d074133-6894-47dd-914c-bf173663afd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:14:18.400882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d074133-6894-47dd-914c-bf173663afd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:18.401167 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d074133-6894-47dd-914c-bf173663afd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:18.401446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d074133-6894-47dd-914c-bf173663afd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:18.410140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d074133-6894-47dd-914c-bf173663afd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:14:18.411565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d074133-6894-47dd-914c-bf173663afd6 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:14:18.421613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 747/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:14:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:14:20.560931 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa22214-1a6e-4e79-94f7-9e5b4890840d None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:20.561490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa22214-1a6e-4e79-94f7-9e5b4890840d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:20.561490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa22214-1a6e-4e79-94f7-9e5b4890840d None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:20.562872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 738/1485] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:14:20.567861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b000-ee43-4d81-b1ff-f5b36da1a1a2 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:20.568290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b000-ee43-4d81-b1ff-f5b36da1a1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:20.568452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9b000-ee43-4d81-b1ff-f5b36da1a1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:20.587000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-89f9b000-ee43-4d81-b1ff-f5b36da1a1a2 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:14:20.588226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-89f9b000-ee43-4d81-b1ff-f5b36da1a1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:14:20.595727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 748/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:14:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:14:20.633219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa22214-1a6e-4e79-94f7-9e5b4890840d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:14:20.643794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa22214-1a6e-4e79-94f7-9e5b4890840d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:14:20.647289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d936a7-36b3-48d7-b6ef-727ead572ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:14:20.647926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d936a7-36b3-48d7-b6ef-727ead572ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:20.648260 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d936a7-36b3-48d7-b6ef-727ead572ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:20.648541 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d936a7-36b3-48d7-b6ef-727ead572ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:20.657250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d936a7-36b3-48d7-b6ef-727ead572ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:14:20.658683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d936a7-36b3-48d7-b6ef-727ead572ed5 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:14:20.668733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 739/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:14:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:14:22.856053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d34b29-901b-4d7f-b184-89d3fc3d9b90 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:22.856616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d34b29-901b-4d7f-b184-89d3fc3d9b90 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:22.856715 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d34b29-901b-4d7f-b184-89d3fc3d9b90 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:22.858034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 749/1488] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:14:22.863250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84237d27-1e44-4727-b1c7-52679ffca7e8 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:22.863676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84237d27-1e44-4727-b1c7-52679ffca7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:22.863792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84237d27-1e44-4727-b1c7-52679ffca7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:22.880945 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-84237d27-1e44-4727-b1c7-52679ffca7e8 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:14:22.882618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-84237d27-1e44-4727-b1c7-52679ffca7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:14:22.890408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 740/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:14:22.930209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d34b29-901b-4d7f-b184-89d3fc3d9b90 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:14:22.940508 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d34b29-901b-4d7f-b184-89d3fc3d9b90 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:14:22.943885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ebd48a-d69c-4075-913f-d42cfc405fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:14:22.944551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebd48a-d69c-4075-913f-d42cfc405fce service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:22.945318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebd48a-d69c-4075-913f-d42cfc405fce service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:22.945589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebd48a-d69c-4075-913f-d42cfc405fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:22.954226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ebd48a-d69c-4075-913f-d42cfc405fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:14:22.955638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ebd48a-d69c-4075-913f-d42cfc405fce service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:14:22.965594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 750/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:14:25.240244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646a7c7-b16f-4a6c-ae49-bccf023d6080 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:25.240845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646a7c7-b16f-4a6c-ae49-bccf023d6080 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:25.240845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646a7c7-b16f-4a6c-ae49-bccf023d6080 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:25.242761 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 741/1491] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jun 17 07:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:14:25.247790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa367b4-7b8c-4412-8d6f-236c9dfab77c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:25.248208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa367b4-7b8c-4412-8d6f-236c9dfab77c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:25.248374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa367b4-7b8c-4412-8d6f-236c9dfab77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:25.265725 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-8aa367b4-7b8c-4412-8d6f-236c9dfab77c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:14:25.266932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-8aa367b4-7b8c-4412-8d6f-236c9dfab77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:14:25.274647 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 751/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jun 17 07:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:14:25.313043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646a7c7-b16f-4a6c-ae49-bccf023d6080 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:14:25.323582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646a7c7-b16f-4a6c-ae49-bccf023d6080 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:14:25.327024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de6875-8b18-4563-b112-b84be7784d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:14:25.327652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de6875-8b18-4563-b112-b84be7784d76 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:14:25.327934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de6875-8b18-4563-b112-b84be7784d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:14:25.328243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de6875-8b18-4563-b112-b84be7784d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:14:25.336992 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de6875-8b18-4563-b112-b84be7784d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:14:25.338402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de6875-8b18-4563-b112-b84be7784d76 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:14:25.347913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 742/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:14:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:14:35.260775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=33 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:14:35.343762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=33 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:15:55.914332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd673769-26a1-42b4-b825-f3ab3af11fee None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:55.915186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd673769-26a1-42b4-b825-f3ab3af11fee None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:55.915186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd673769-26a1-42b4-b825-f3ab3af11fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:55.932710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-dd673769-26a1-42b4-b825-f3ab3af11fee None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:15:55.933882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-dd673769-26a1-42b4-b825-f3ab3af11fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:15:55.943299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 752/1494] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:15:55.952436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7012ce-cb06-44e3-b5c8-83543fc4c0f8 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:55.952936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7012ce-cb06-44e3-b5c8-83543fc4c0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:55.953033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7012ce-cb06-44e3-b5c8-83543fc4c0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:55.970835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-3c7012ce-cb06-44e3-b5c8-83543fc4c0f8 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:15:55.972022 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3c7012ce-cb06-44e3-b5c8-83543fc4c0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:15:55.980379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 743/1495] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:15:55.984705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd673769-26a1-42b4-b825-f3ab3af11fee None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:15:55.994072 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd673769-26a1-42b4-b825-f3ab3af11fee None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:15:55.998744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992925f6-c669-4bfb-a4fb-9897013d1115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:15:55.999401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992925f6-c669-4bfb-a4fb-9897013d1115 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:55.999686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992925f6-c669-4bfb-a4fb-9897013d1115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:55.999962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992925f6-c669-4bfb-a4fb-9897013d1115 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.000714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992925f6-c669-4bfb-a4fb-9897013d1115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:15:56.001023 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992925f6-c669-4bfb-a4fb-9897013d1115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:15:56.002063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: WARNING py.warnings [None req-992925f6-c669-4bfb-a4fb-9897013d1115 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:15:56.002063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: warnings.warn(msg) Jun 17 07:15:56.002063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: Jun 17 07:15:56.002956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992925f6-c669-4bfb-a4fb-9897013d1115 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:15:56.014780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 753/1496] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:15:56.023595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7012ce-cb06-44e3-b5c8-83543fc4c0f8 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:15:56.032567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7012ce-cb06-44e3-b5c8-83543fc4c0f8 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:15:56.037153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04856a-b4a4-4f88-ad38-a48db4f6518f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:15:56.037795 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04856a-b4a4-4f88-ad38-a48db4f6518f admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.038080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04856a-b4a4-4f88-ad38-a48db4f6518f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.038360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04856a-b4a4-4f88-ad38-a48db4f6518f admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.039048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04856a-b4a4-4f88-ad38-a48db4f6518f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:15:56.041113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: WARNING py.warnings [None req-2f04856a-b4a4-4f88-ad38-a48db4f6518f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:15:56.041113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: warnings.warn(msg) Jun 17 07:15:56.041113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: Jun 17 07:15:56.042321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04856a-b4a4-4f88-ad38-a48db4f6518f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:15:56.076864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 744/1497] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:15:56 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:15:56.081107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992925f6-c669-4bfb-a4fb-9897013d1115 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:15:56.091252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992925f6-c669-4bfb-a4fb-9897013d1115 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:15:56.145996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a50fad-c821-4949-8315-eaccbc886607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:15:56.146666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a50fad-c821-4949-8315-eaccbc886607 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.146955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a50fad-c821-4949-8315-eaccbc886607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.147236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a50fad-c821-4949-8315-eaccbc886607 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.147930 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a50fad-c821-4949-8315-eaccbc886607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:15:56.148974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: WARNING py.warnings [None req-15a50fad-c821-4949-8315-eaccbc886607 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:15:56.148974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: warnings.warn(msg) Jun 17 07:15:56.148974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: Jun 17 07:15:56.150191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a50fad-c821-4949-8315-eaccbc886607 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:15:56.170953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 754/1498] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:15:56 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:15:56.175114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04856a-b4a4-4f88-ad38-a48db4f6518f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:15:56.185174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04856a-b4a4-4f88-ad38-a48db4f6518f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:15:56.189614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f85436-0ec5-463b-89ce-dc07ae04bfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:15:56.190259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f85436-0ec5-463b-89ce-dc07ae04bfdb admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.190541 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f85436-0ec5-463b-89ce-dc07ae04bfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.190813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f85436-0ec5-463b-89ce-dc07ae04bfdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.191519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f85436-0ec5-463b-89ce-dc07ae04bfdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:15:56.192589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: WARNING py.warnings [None req-64f85436-0ec5-463b-89ce-dc07ae04bfdb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:15:56.192589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: warnings.warn(msg) Jun 17 07:15:56.192589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: Jun 17 07:15:56.193968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f85436-0ec5-463b-89ce-dc07ae04bfdb admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:15:56.206343 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 745/1499] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:15:56.211162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a50fad-c821-4949-8315-eaccbc886607 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:15:56.222969 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a50fad-c821-4949-8315-eaccbc886607 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:15:56.227573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa825c4-a7b0-4d2d-83f5-4516f7cbada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:15:56.228246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa825c4-a7b0-4d2d-83f5-4516f7cbada7 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.228532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa825c4-a7b0-4d2d-83f5-4516f7cbada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.228816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa825c4-a7b0-4d2d-83f5-4516f7cbada7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.229523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa825c4-a7b0-4d2d-83f5-4516f7cbada7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:15:56.230515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: WARNING py.warnings [None req-7fa825c4-a7b0-4d2d-83f5-4516f7cbada7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:15:56.230515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: warnings.warn(msg) Jun 17 07:15:56.230515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: Jun 17 07:15:56.231392 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa825c4-a7b0-4d2d-83f5-4516f7cbada7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:15:56.244431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 755/1500] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:15:56.248612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f85436-0ec5-463b-89ce-dc07ae04bfdb admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:15:56.258743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f85436-0ec5-463b-89ce-dc07ae04bfdb admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:15:56.263144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ce97a-8e80-4ca1-942b-427a4bdea1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:15:56.264033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ce97a-8e80-4ca1-942b-427a4bdea1e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.264349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ce97a-8e80-4ca1-942b-427a4bdea1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.264629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ce97a-8e80-4ca1-942b-427a4bdea1e8 admin admin] PATH_INFO: `/v3/projects/93ad89d5b4154677a71e98ba7e2d2f7c/users/4915c9cf8ac0442889223266fb6ceaf1/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.280155 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ce97a-8e80-4ca1-942b-427a4bdea1e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ad89d5b4154677a71e98ba7e2d2f7c, user_id=4915c9cf8ac0442889223266fb6ceaf1, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:15:56.281170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: WARNING py.warnings [None req-102ce97a-8e80-4ca1-942b-427a4bdea1e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:15:56.281170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: warnings.warn(msg) Jun 17 07:15:56.281170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: Jun 17 07:15:56.283089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ce97a-8e80-4ca1-942b-427a4bdea1e8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:15:56.295782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 746/1501] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:15:56 2022] PUT /identity/v3/projects/93ad89d5b4154677a71e98ba7e2d2f7c/users/4915c9cf8ac0442889223266fb6ceaf1/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:15:56.306730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003c33e-1531-4dad-b166-d5639d61f73c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.307050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003c33e-1531-4dad-b166-d5639d61f73c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.307330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003c33e-1531-4dad-b166-d5639d61f73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.334338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-7003c33e-1531-4dad-b166-d5639d61f73c None None] MFA Rules not processed for user `4915c9cf8ac0442889223266fb6ceaf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:15:56.335547 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7003c33e-1531-4dad-b166-d5639d61f73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:15:56.397737 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 756/1502] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:15:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:15:56.410720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519b29c-866c-401b-a8f6-5f91063c1181 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.411013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519b29c-866c-401b-a8f6-5f91063c1181 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.411294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519b29c-866c-401b-a8f6-5f91063c1181 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.412725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 747/1503] 10.222.0.33 () {54 vars in 963 bytes} [Fri Jun 17 07:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:15:56.417784 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577b4a4-0370-4d02-9841-d2103b13176b None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.418073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577b4a4-0370-4d02-9841-d2103b13176b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.418348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577b4a4-0370-4d02-9841-d2103b13176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.436036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-9577b4a4-0370-4d02-9841-d2103b13176b None None] MFA Rules not processed for user `3769534c610b448db693f7c5d6133e65`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:15:56.437241 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9577b4a4-0370-4d02-9841-d2103b13176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:15:56.478531 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:15:56.485878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 757/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Fri Jun 17 07:15:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:15:56.493991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519b29c-866c-401b-a8f6-5f91063c1181 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:15:56.504407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519b29c-866c-401b-a8f6-5f91063c1181 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:15:56.508568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222c3fe-7b89-4ce7-84b4-bc95d7f6ccaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:15:56.509198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222c3fe-7b89-4ce7-84b4-bc95d7f6ccaf service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.509483 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222c3fe-7b89-4ce7-84b4-bc95d7f6ccaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.509763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222c3fe-7b89-4ce7-84b4-bc95d7f6ccaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.517993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222c3fe-7b89-4ce7-84b4-bc95d7f6ccaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:15:56.519417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222c3fe-7b89-4ce7-84b4-bc95d7f6ccaf service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:15:56.528140 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 748/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:15:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:15:56.570125 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a251a45-4e2c-4c9b-a81d-71bb2a4d6492 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.570415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a251a45-4e2c-4c9b-a81d-71bb2a4d6492 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.570695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a251a45-4e2c-4c9b-a81d-71bb2a4d6492 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.572012 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 758/1506] 10.222.0.33 () {52 vars in 829 bytes} [Fri Jun 17 07:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:15:56.575735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222c3fe-7b89-4ce7-84b4-bc95d7f6ccaf service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:15:56.638339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222c3fe-7b89-4ce7-84b4-bc95d7f6ccaf service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:15:56.642833 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:15:56.643695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.643979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.644287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] PATH_INFO: `/v3/users/4915c9cf8ac0442889223266fb6ceaf1` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.645461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4915c9cf8ac0442889223266fb6ceaf1)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:15:56.646974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: WARNING py.warnings [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:15:56.646974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: warnings.warn(msg) Jun 17 07:15:56.646974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: Jun 17 07:15:56.648237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:15:56.712786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4915c9cf8ac0442889223266fb6ceaf1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:15:56.721702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4915c9cf8ac0442889223266fb6ceaf1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:15:56.722406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4915c9cf8ac0442889223266fb6ceaf1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:15:56.740818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4915c9cf8ac0442889223266fb6ceaf1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:15:56.741530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4915c9cf8ac0442889223266fb6ceaf1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:15:56.750331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:15:56.761797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 749/1507] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:15:56 2022] DELETE /identity/v3/users/4915c9cf8ac0442889223266fb6ceaf1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:15:56.766167 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e0644-fed6-4d43-8333-36194b306c39 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.766482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e0644-fed6-4d43-8333-36194b306c39 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.766762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e0644-fed6-4d43-8333-36194b306c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.784307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-562e0644-fed6-4d43-8333-36194b306c39 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:15:56.785504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-562e0644-fed6-4d43-8333-36194b306c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:15:56.842729 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 759/1508] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:15:56.852115 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:15:56.853664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:15:56.911032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93603c-e68b-41df-a7bf-660f20b2330d admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:15:56.914926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194b3bb-7d9e-4bc0-a198-fc79fb4360be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:15:56.915559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194b3bb-7d9e-4bc0-a198-fc79fb4360be service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:56.915845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194b3bb-7d9e-4bc0-a198-fc79fb4360be service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:56.916157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194b3bb-7d9e-4bc0-a198-fc79fb4360be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:56.924929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194b3bb-7d9e-4bc0-a198-fc79fb4360be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:15:56.926367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194b3bb-7d9e-4bc0-a198-fc79fb4360be service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:15:56.935871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 750/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:15:57.251640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e0644-fed6-4d43-8333-36194b306c39 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:15:57.253109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-562e0644-fed6-4d43-8333-36194b306c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:15:57.264907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e0644-fed6-4d43-8333-36194b306c39 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:15:57.270068 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:15:57.271447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:15:57.271734 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:15:57.276560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] PATH_INFO: `/v3/projects/93ad89d5b4154677a71e98ba7e2d2f7c` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:15:57.277753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ad89d5b4154677a71e98ba7e2d2f7c)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:15:57.279028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: WARNING py.warnings [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:15:57.279028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: warnings.warn(msg) Jun 17 07:15:57.279028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: Jun 17 07:15:57.280016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:15:57.342193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ad89d5b4154677a71e98ba7e2d2f7c', 'request_id': 'req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:15:57.352004 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ad89d5b4154677a71e98ba7e2d2f7c', 'request_id': 'req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:15:57.365637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ad89d5b4154677a71e98ba7e2d2f7c', 'request_id': 'req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:15:57.376277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] The token cache is being invalidate because project 93ad89d5b4154677a71e98ba7e2d2f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:15:57.376565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:15:57.377194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-0d2bf26b-5c2d-4fd5-aa27-d2ff3613f5ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:15:57.378824 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 760/1510] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:15:57 2022] DELETE /identity/v3/projects/93ad89d5b4154677a71e98ba7e2d2f7c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:16:06.931594 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=29 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:16:07.373291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:17:39.280051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5a80a-cb33-4123-b91d-41017d8b982d None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:39.281188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5a80a-cb33-4123-b91d-41017d8b982d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:39.281188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5a80a-cb33-4123-b91d-41017d8b982d None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:39.282524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 07:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:17:39.287502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772a14f-cda5-4a5a-8918-66bedf6e5acd None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:39.287970 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772a14f-cda5-4a5a-8918-66bedf6e5acd None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:39.288101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772a14f-cda5-4a5a-8918-66bedf6e5acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:39.306675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-7772a14f-cda5-4a5a-8918-66bedf6e5acd None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:17:39.307917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7772a14f-cda5-4a5a-8918-66bedf6e5acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:17:39.365276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 07:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:17:39.534202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5a80a-cb33-4123-b91d-41017d8b982d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:17:39.535858 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5cf5a80a-cb33-4123-b91d-41017d8b982d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:17:39.607544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5a80a-cb33-4123-b91d-41017d8b982d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:17:39.612944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d07e53-de1a-4a4e-b017-aff7dc14ec62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:17:39.613581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d07e53-de1a-4a4e-b017-aff7dc14ec62 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:39.613868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d07e53-de1a-4a4e-b017-aff7dc14ec62 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:39.614149 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d07e53-de1a-4a4e-b017-aff7dc14ec62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:39.622075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d07e53-de1a-4a4e-b017-aff7dc14ec62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:17:39.623656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d07e53-de1a-4a4e-b017-aff7dc14ec62 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:17:39.632826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 752/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:17:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:17:40.501617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2ec7e-0977-480f-a163-f77c4ebfc461 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:40.502173 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2ec7e-0977-480f-a163-f77c4ebfc461 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:40.502278 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2ec7e-0977-480f-a163-f77c4ebfc461 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:40.503642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 07:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:17:40.508718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc71cbd-9c82-40f2-9733-9bc1401e41e4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:40.509149 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc71cbd-9c82-40f2-9733-9bc1401e41e4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:40.509291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc71cbd-9c82-40f2-9733-9bc1401e41e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:40.527171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-3fc71cbd-9c82-40f2-9733-9bc1401e41e4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:17:40.528422 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3fc71cbd-9c82-40f2-9733-9bc1401e41e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:17:40.536250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 07:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:17:40.692656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2ec7e-0977-480f-a163-f77c4ebfc461 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:17:40.707457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2ec7e-0977-480f-a163-f77c4ebfc461 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:17:40.712778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b368893-4287-4630-95ab-a592cfd4a94e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:17:40.713413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b368893-4287-4630-95ab-a592cfd4a94e service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:40.713699 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b368893-4287-4630-95ab-a592cfd4a94e service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:40.713975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b368893-4287-4630-95ab-a592cfd4a94e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:40.722461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b368893-4287-4630-95ab-a592cfd4a94e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:17:40.724048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b368893-4287-4630-95ab-a592cfd4a94e service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:17:40.733861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 763/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:17:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:17:40.932705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dace7a6-b356-49b4-914b-151a61f994b5 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:40.933021 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dace7a6-b356-49b4-914b-151a61f994b5 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:40.933305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dace7a6-b356-49b4-914b-151a61f994b5 None None] PATH_INFO: `/v3` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:40.934623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 754/1517] 10.222.0.16 () {54 vars in 925 bytes} [Fri Jun 17 07:17:40 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:17:40.941249 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b58d6-6d9e-438b-b46c-4a8eef67df3c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:40.941544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b58d6-6d9e-438b-b46c-4a8eef67df3c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:40.941829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b58d6-6d9e-438b-b46c-4a8eef67df3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:40.960141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-070b58d6-6d9e-438b-b46c-4a8eef67df3c None None] MFA Rules not processed for user `b8cf02a52d354517b363c0aaf477550c`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:17:40.961295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-070b58d6-6d9e-438b-b46c-4a8eef67df3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:17:41.021638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 764/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Jun 17 07:17:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:17:44.740748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7f38f-484f-48d4-9c2b-e195564a0cda None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:44.741313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7f38f-484f-48d4-9c2b-e195564a0cda None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:44.741313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7f38f-484f-48d4-9c2b-e195564a0cda None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:44.742725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 755/1519] 10.222.0.33 () {54 vars in 960 bytes} [Fri Jun 17 07:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:17:44.747776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463756a9-7ea8-479b-98e5-ad5cb53e742a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:44.748249 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463756a9-7ea8-479b-98e5-ad5cb53e742a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:44.748362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463756a9-7ea8-479b-98e5-ad5cb53e742a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:44.766643 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-463756a9-7ea8-479b-98e5-ad5cb53e742a None None] MFA Rules not processed for user `b8cf02a52d354517b363c0aaf477550c`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:17:44.767843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-463756a9-7ea8-479b-98e5-ad5cb53e742a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:17:44.776877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 765/1520] 10.222.0.33 () {58 vars in 1057 bytes} [Fri Jun 17 07:17:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:17:44.784580 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7f38f-484f-48d4-9c2b-e195564a0cda None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:17:44.799374 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7f38f-484f-48d4-9c2b-e195564a0cda None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:17:44.806210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973faa5-f979-495d-9e5c-7980e4c99932 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8cf02a52d354517b363c0aaf477550c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:17:44.806877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973faa5-f979-495d-9e5c-7980e4c99932 service placement] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:44.807145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973faa5-f979-495d-9e5c-7980e4c99932 service placement] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:44.807420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973faa5-f979-495d-9e5c-7980e4c99932 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:44.816860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973faa5-f979-495d-9e5c-7980e4c99932 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:17:44.818504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973faa5-f979-495d-9e5c-7980e4c99932 service placement] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:17:44.828416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 756/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:17:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:17:52.050293 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a894-81b9-4a04-b350-48c59c1f5d1c None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:52.050817 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a894-81b9-4a04-b350-48c59c1f5d1c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:52.050817 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a894-81b9-4a04-b350-48c59c1f5d1c None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:52.052211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 07:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:17:52.056988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf0f11-a28a-4964-bca9-2e4b30bf625b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:52.057431 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf0f11-a28a-4964-bca9-2e4b30bf625b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:52.057561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf0f11-a28a-4964-bca9-2e4b30bf625b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:52.075703 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-bcaf0f11-a28a-4964-bca9-2e4b30bf625b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:17:52.077554 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-bcaf0f11-a28a-4964-bca9-2e4b30bf625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:17:52.085461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 07:17:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:17:52.238951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6a894-81b9-4a04-b350-48c59c1f5d1c None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:17:52.254446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6a894-81b9-4a04-b350-48c59c1f5d1c None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:17:52.259693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb7c20-06b2-4554-b280-257403a77e6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:17:52.260397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb7c20-06b2-4554-b280-257403a77e6b service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:52.260683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb7c20-06b2-4554-b280-257403a77e6b service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:52.260959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb7c20-06b2-4554-b280-257403a77e6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:52.269777 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb7c20-06b2-4554-b280-257403a77e6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:17:52.271361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb7c20-06b2-4554-b280-257403a77e6b service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:17:52.280888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 767/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:17:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:17:53.146448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f727c-05a1-435a-821b-c896357e6a69 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:53.147036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f727c-05a1-435a-821b-c896357e6a69 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:53.147036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f727c-05a1-435a-821b-c896357e6a69 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:53.148456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 07:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:17:53.153126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179818b1-b15f-4b8a-92b7-264fb7907af0 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:53.153554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179818b1-b15f-4b8a-92b7-264fb7907af0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:53.153665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179818b1-b15f-4b8a-92b7-264fb7907af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:53.172390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-179818b1-b15f-4b8a-92b7-264fb7907af0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:17:53.173567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-179818b1-b15f-4b8a-92b7-264fb7907af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:17:53.181312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 07:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:17:53.332793 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f727c-05a1-435a-821b-c896357e6a69 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:17:53.347593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f727c-05a1-435a-821b-c896357e6a69 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:17:53.353034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4557ebc-c17d-4dc3-ac97-f84bdbb776f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:17:53.353673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4557ebc-c17d-4dc3-ac97-f84bdbb776f3 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:53.353970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4557ebc-c17d-4dc3-ac97-f84bdbb776f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:53.354255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4557ebc-c17d-4dc3-ac97-f84bdbb776f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:53.362837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4557ebc-c17d-4dc3-ac97-f84bdbb776f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:17:53.364487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4557ebc-c17d-4dc3-ac97-f84bdbb776f3 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:17:53.373760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 759/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:17:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:17:57.419747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dab8b0-1d2a-4268-b1d9-a866dcaf1d6b None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:57.420854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dab8b0-1d2a-4268-b1d9-a866dcaf1d6b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:57.420991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dab8b0-1d2a-4268-b1d9-a866dcaf1d6b None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:57.422380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:17:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:17:57.427379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c57c7-4c8d-4dd2-b2c6-aac5ca2e86a7 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:57.427791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c57c7-4c8d-4dd2-b2c6-aac5ca2e86a7 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:57.427936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c57c7-4c8d-4dd2-b2c6-aac5ca2e86a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:57.445358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c14c57c7-4c8d-4dd2-b2c6-aac5ca2e86a7 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:17:57.446546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c14c57c7-4c8d-4dd2-b2c6-aac5ca2e86a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:17:57.454768 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:17:57.491520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dab8b0-1d2a-4268-b1d9-a866dcaf1d6b None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:17:57.493231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-46dab8b0-1d2a-4268-b1d9-a866dcaf1d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:17:57.551211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dab8b0-1d2a-4268-b1d9-a866dcaf1d6b None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:17:57.554691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d9c87-7fca-474c-9836-d1590ed52cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:17:57.555336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d9c87-7fca-474c-9836-d1590ed52cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:57.555626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d9c87-7fca-474c-9836-d1590ed52cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:57.555907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d9c87-7fca-474c-9836-d1590ed52cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:57.565004 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d9c87-7fca-474c-9836-d1590ed52cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:17:57.566431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d9c87-7fca-474c-9836-d1590ed52cf2 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:17:57.571264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:17:57.577258 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 770/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:17:59.646393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdfdeb-f09e-459b-82c2-347f8282610d None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:59.646908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdfdeb-f09e-459b-82c2-347f8282610d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:59.646908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdfdeb-f09e-459b-82c2-347f8282610d None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:59.648291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:17:59.653272 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7359e-2ba5-4cc2-8744-c3aec8206548 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:59.653663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7359e-2ba5-4cc2-8744-c3aec8206548 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:59.653830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7359e-2ba5-4cc2-8744-c3aec8206548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:59.671528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-bfb7359e-2ba5-4cc2-8744-c3aec8206548 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:17:59.672753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-bfb7359e-2ba5-4cc2-8744-c3aec8206548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:17:59.680337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:17:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:17:59.715760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdfdeb-f09e-459b-82c2-347f8282610d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:17:59.726547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdfdeb-f09e-459b-82c2-347f8282610d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:17:59.730039 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa168353-4aed-4e30-8b78-cc52e858da1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:17:59.730670 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa168353-4aed-4e30-8b78-cc52e858da1b service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:17:59.730954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa168353-4aed-4e30-8b78-cc52e858da1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:17:59.731234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa168353-4aed-4e30-8b78-cc52e858da1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:17:59.740567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa168353-4aed-4e30-8b78-cc52e858da1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:17:59.741990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa168353-4aed-4e30-8b78-cc52e858da1b service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:17:59.751635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 762/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:17:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:18:03.558088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5cb7f-ad1e-4c95-8f96-5079dd6437ce None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:18:03.558695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5cb7f-ad1e-4c95-8f96-5079dd6437ce None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:18:03.558695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5cb7f-ad1e-4c95-8f96-5079dd6437ce None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:18:03.560022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:18:03.565084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d659b2f-8147-4931-911b-92f2e75a11ec None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:18:03.565533 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d659b2f-8147-4931-911b-92f2e75a11ec None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:18:03.565639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d659b2f-8147-4931-911b-92f2e75a11ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:18:03.585510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6d659b2f-8147-4931-911b-92f2e75a11ec None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:18:03.586733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6d659b2f-8147-4931-911b-92f2e75a11ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:18:03.595171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:18:03.858197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5cb7f-ad1e-4c95-8f96-5079dd6437ce None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:18:03.859878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3fb5cb7f-ad1e-4c95-8f96-5079dd6437ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:18:03.916375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5cb7f-ad1e-4c95-8f96-5079dd6437ce None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:18:03.920389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eda7f6-1ab0-4483-bbd3-7f036358e11a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:18:03.921031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eda7f6-1ab0-4483-bbd3-7f036358e11a service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:18:03.921316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eda7f6-1ab0-4483-bbd3-7f036358e11a service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:18:03.921590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eda7f6-1ab0-4483-bbd3-7f036358e11a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:18:03.930509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eda7f6-1ab0-4483-bbd3-7f036358e11a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:18:03.931927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eda7f6-1ab0-4483-bbd3-7f036358e11a service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:18:03.941182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 773/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:18:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:18:06.072995 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce093145-a9dc-488d-9d72-2bb080921afa None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:18:06.073554 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce093145-a9dc-488d-9d72-2bb080921afa None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:18:06.073554 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce093145-a9dc-488d-9d72-2bb080921afa None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:18:06.074837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 07:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:18:06.080001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97194cb-b438-4b43-bb6c-87e73894b93f None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:18:06.080473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97194cb-b438-4b43-bb6c-87e73894b93f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:18:06.080591 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97194cb-b438-4b43-bb6c-87e73894b93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:18:06.098703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f97194cb-b438-4b43-bb6c-87e73894b93f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:18:06.099890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f97194cb-b438-4b43-bb6c-87e73894b93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:18:06.108116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 07:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:18:06.371644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce093145-a9dc-488d-9d72-2bb080921afa None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:18:06.374011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ce093145-a9dc-488d-9d72-2bb080921afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:18:06.387109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce093145-a9dc-488d-9d72-2bb080921afa None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:18:06.391374 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d894c7-f644-4efb-9054-9f9271810f8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:18:06.392017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d894c7-f644-4efb-9054-9f9271810f8a service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:18:06.392323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d894c7-f644-4efb-9054-9f9271810f8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:18:06.392599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d894c7-f644-4efb-9054-9f9271810f8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:18:06.401681 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d894c7-f644-4efb-9054-9f9271810f8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:18:06.403097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d894c7-f644-4efb-9054-9f9271810f8a service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:18:06.412837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 765/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:18:16.093833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=32 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:18:16.408240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=70 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:35.039444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911e568-85bd-41c8-b0c4-60b642e6bc16 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.040118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911e568-85bd-41c8-b0c4-60b642e6bc16 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.040118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911e568-85bd-41c8-b0c4-60b642e6bc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.063702 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-9911e568-85bd-41c8-b0c4-60b642e6bc16 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.065080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9911e568-85bd-41c8-b0c4-60b642e6bc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.076205 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 775/1540] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:35.086967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db31144-173d-40de-a2a6-5f2f0b8b4d78 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.087537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db31144-173d-40de-a2a6-5f2f0b8b4d78 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.087537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db31144-173d-40de-a2a6-5f2f0b8b4d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.094706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193afa6-a53d-482f-a4f7-266b74c4a6d3 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.095001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193afa6-a53d-482f-a4f7-266b74c4a6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.095284 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193afa6-a53d-482f-a4f7-266b74c4a6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.109777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6db31144-173d-40de-a2a6-5f2f0b8b4d78 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.110999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6db31144-173d-40de-a2a6-5f2f0b8b4d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.125178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-d193afa6-a53d-482f-a4f7-266b74c4a6d3 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.126381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d193afa6-a53d-482f-a4f7-266b74c4a6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.129263 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 766/1541] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:35.133745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db31144-173d-40de-a2a6-5f2f0b8b4d78 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:35.148967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db31144-173d-40de-a2a6-5f2f0b8b4d78 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:35.154422 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 776/1542] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:35.157310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebeebe-3f90-4523-aba5-cc1f6aae4602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:35.158007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebeebe-3f90-4523-aba5-cc1f6aae4602 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.158296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebeebe-3f90-4523-aba5-cc1f6aae4602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.158719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebeebe-3f90-4523-aba5-cc1f6aae4602 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.159469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebeebe-3f90-4523-aba5-cc1f6aae4602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:19:35.159782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebeebe-3f90-4523-aba5-cc1f6aae4602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:35.160848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: WARNING py.warnings [None req-b9ebeebe-3f90-4523-aba5-cc1f6aae4602 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:19:35.160848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: warnings.warn(msg) Jun 17 07:19:35.160848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: Jun 17 07:19:35.162017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebeebe-3f90-4523-aba5-cc1f6aae4602 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:35.165101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2bced-7ce2-4792-943d-2719d2242fc8 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.165396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2bced-7ce2-4792-943d-2719d2242fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.165828 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2bced-7ce2-4792-943d-2719d2242fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.176988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 767/1543] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:19:35.185537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebeebe-3f90-4523-aba5-cc1f6aae4602 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:35.196256 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebeebe-3f90-4523-aba5-cc1f6aae4602 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:35.210400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c7f2bced-7ce2-4792-943d-2719d2242fc8 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.211655 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c7f2bced-7ce2-4792-943d-2719d2242fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.213129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524a8d7-bb2b-421f-b68d-9f85857ee667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:35.214062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524a8d7-bb2b-421f-b68d-9f85857ee667 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.214358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524a8d7-bb2b-421f-b68d-9f85857ee667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.214644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524a8d7-bb2b-421f-b68d-9f85857ee667 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.215357 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524a8d7-bb2b-421f-b68d-9f85857ee667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:35.217314 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524a8d7-bb2b-421f-b68d-9f85857ee667 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:35.236309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 777/1544] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:35.241652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2bced-7ce2-4792-943d-2719d2242fc8 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:35.259049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2bced-7ce2-4792-943d-2719d2242fc8 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:35.266887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 768/1545] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:35.269090 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0caaf20-168d-4dbb-94c2-c36496b37ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:35.269900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caaf20-168d-4dbb-94c2-c36496b37ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.270192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caaf20-168d-4dbb-94c2-c36496b37ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.270478 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caaf20-168d-4dbb-94c2-c36496b37ec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.271141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524a8d7-bb2b-421f-b68d-9f85857ee667 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:35.271268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0caaf20-168d-4dbb-94c2-c36496b37ec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:19:35.271667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0caaf20-168d-4dbb-94c2-c36496b37ec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:35.273332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0caaf20-168d-4dbb-94c2-c36496b37ec8 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:35.299313 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 778/1546] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:19:35.307931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0caaf20-168d-4dbb-94c2-c36496b37ec8 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:35.328506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0caaf20-168d-4dbb-94c2-c36496b37ec8 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:35.386733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524a8d7-bb2b-421f-b68d-9f85857ee667 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:35.410400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aaa1cc-15c4-4ca5-bd0c-6beef33f41eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:35.411378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaa1cc-15c4-4ca5-bd0c-6beef33f41eb admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.411673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaa1cc-15c4-4ca5-bd0c-6beef33f41eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.411956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaa1cc-15c4-4ca5-bd0c-6beef33f41eb admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.412871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaa1cc-15c4-4ca5-bd0c-6beef33f41eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:35.413925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: WARNING py.warnings [None req-23aaa1cc-15c4-4ca5-bd0c-6beef33f41eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:19:35.413925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: warnings.warn(msg) Jun 17 07:19:35.413925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: Jun 17 07:19:35.415341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaa1cc-15c4-4ca5-bd0c-6beef33f41eb admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:35.455781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cc182-c136-488b-8382-3596aa77ed15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:35.456590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cc182-c136-488b-8382-3596aa77ed15 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.456745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 769/1547] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:35.456845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cc182-c136-488b-8382-3596aa77ed15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.457097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cc182-c136-488b-8382-3596aa77ed15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.457843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cc182-c136-488b-8382-3596aa77ed15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:35.458928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: WARNING py.warnings [None req-c32cc182-c136-488b-8382-3596aa77ed15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:19:35.458928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: warnings.warn(msg) Jun 17 07:19:35.458928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: Jun 17 07:19:35.460625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cc182-c136-488b-8382-3596aa77ed15 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:35.460760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aaa1cc-15c4-4ca5-bd0c-6beef33f41eb admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:35.474460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aaa1cc-15c4-4ca5-bd0c-6beef33f41eb admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:35.491872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4d85c-5afc-4780-b295-6daa02946478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:35.492898 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4d85c-5afc-4780-b295-6daa02946478 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.493101 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4d85c-5afc-4780-b295-6daa02946478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.493349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4d85c-5afc-4780-b295-6daa02946478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.494637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4d85c-5afc-4780-b295-6daa02946478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:35.496323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4d85c-5afc-4780-b295-6daa02946478 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:35.517127 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 770/1548] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:19:35.519541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 779/1549] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/projects => generated 405 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:35.521419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4d85c-5afc-4780-b295-6daa02946478 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:35.523402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cc182-c136-488b-8382-3596aa77ed15 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:35.537465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4d85c-5afc-4780-b295-6daa02946478 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:35.618760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d237689-2fc6-4bf9-800c-5fc2cd9105bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:35.621725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d237689-2fc6-4bf9-800c-5fc2cd9105bf admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.622010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d237689-2fc6-4bf9-800c-5fc2cd9105bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.622289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d237689-2fc6-4bf9-800c-5fc2cd9105bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.623019 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d237689-2fc6-4bf9-800c-5fc2cd9105bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:35.624694 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d237689-2fc6-4bf9-800c-5fc2cd9105bf admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:35.639115 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 771/1550] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:35.640811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e5fe9-ac0d-49e6-bed8-8ec5ce356dc3 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.641129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e5fe9-ac0d-49e6-bed8-8ec5ce356dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.641407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e5fe9-ac0d-49e6-bed8-8ec5ce356dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.651735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cc182-c136-488b-8382-3596aa77ed15 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:35.657218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931ad44-147c-4ccf-9d68-318a38be615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:35.657895 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931ad44-147c-4ccf-9d68-318a38be615f admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.658183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931ad44-147c-4ccf-9d68-318a38be615f admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.658465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931ad44-147c-4ccf-9d68-318a38be615f admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.659024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-705e5fe9-ac0d-49e6-bed8-8ec5ce356dc3 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.659215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931ad44-147c-4ccf-9d68-318a38be615f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:35.660284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-705e5fe9-ac0d-49e6-bed8-8ec5ce356dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.661389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931ad44-147c-4ccf-9d68-318a38be615f admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:35.668806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 772/1551] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:35.670474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83ac23-566f-4d63-814b-8b574219d332 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.670962 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83ac23-566f-4d63-814b-8b574219d332 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.671246 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83ac23-566f-4d63-814b-8b574219d332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.695377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 780/1552] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/users => generated 303 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:35.695377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd82540-98d4-47d2-a700-528d3ea903ff None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.695377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd82540-98d4-47d2-a700-528d3ea903ff None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.695377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd82540-98d4-47d2-a700-528d3ea903ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.696897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-be83ac23-566f-4d63-814b-8b574219d332 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.698716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-be83ac23-566f-4d63-814b-8b574219d332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.718000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3cd82540-98d4-47d2-a700-528d3ea903ff None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.719122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 773/1553] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:35.719286 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3cd82540-98d4-47d2-a700-528d3ea903ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.720525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5afced-2f4f-480f-8c95-62a32f5dddda None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.720811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5afced-2f4f-480f-8c95-62a32f5dddda None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.721086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5afced-2f4f-480f-8c95-62a32f5dddda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.728228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 781/1554] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:35.730521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a24ed-626c-4299-89c5-4e411efa80e3 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.730521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a24ed-626c-4299-89c5-4e411efa80e3 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.730745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a24ed-626c-4299-89c5-4e411efa80e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.741835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-de5afced-2f4f-480f-8c95-62a32f5dddda None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.742973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-de5afced-2f4f-480f-8c95-62a32f5dddda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.749084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-594a24ed-626c-4299-89c5-4e411efa80e3 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.750256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-594a24ed-626c-4299-89c5-4e411efa80e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.751672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 774/1555] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:35.752310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5afced-2f4f-480f-8c95-62a32f5dddda None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:35.759208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 782/1556] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:35.761002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a35c21-47f5-4747-bb70-6569b46f593d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.761297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a35c21-47f5-4747-bb70-6569b46f593d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.761579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a35c21-47f5-4747-bb70-6569b46f593d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.761975 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5afced-2f4f-480f-8c95-62a32f5dddda None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:35.770776 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104c8ba-8d7d-47eb-bef4-5197019eb3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:35.771666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104c8ba-8d7d-47eb-bef4-5197019eb3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.771948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104c8ba-8d7d-47eb-bef4-5197019eb3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.772265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104c8ba-8d7d-47eb-bef4-5197019eb3a4 admin admin] PATH_INFO: `/v3/projects/ab7deafed6f447faade3e3d8d281617a/users/b3ca63f8f8484ddf9d7c72749ba7f482/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.782808 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-29a35c21-47f5-4747-bb70-6569b46f593d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.783523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:35.784195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-29a35c21-47f5-4747-bb70-6569b46f593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.788269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104c8ba-8d7d-47eb-bef4-5197019eb3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7deafed6f447faade3e3d8d281617a, user_id=b3ca63f8f8484ddf9d7c72749ba7f482, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:35.792229 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104c8ba-8d7d-47eb-bef4-5197019eb3a4 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:35.793097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 783/1557] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:35.797295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a35c21-47f5-4747-bb70-6569b46f593d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:35.806382 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 775/1558] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:35 2022] PUT /identity/v3/projects/ab7deafed6f447faade3e3d8d281617a/users/b3ca63f8f8484ddf9d7c72749ba7f482/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:35.807476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a35c21-47f5-4747-bb70-6569b46f593d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:35.808275 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62472504-f73b-4323-86c2-eeddd1873129 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.808576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62472504-f73b-4323-86c2-eeddd1873129 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.808859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62472504-f73b-4323-86c2-eeddd1873129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.826421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:35.827093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-62472504-f73b-4323-86c2-eeddd1873129 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.828456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-62472504-f73b-4323-86c2-eeddd1873129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.867960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249aba13-4022-4802-974c-cca95afbfee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:35.868675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249aba13-4022-4802-974c-cca95afbfee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.869063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249aba13-4022-4802-974c-cca95afbfee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.869890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249aba13-4022-4802-974c-cca95afbfee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.870622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249aba13-4022-4802-974c-cca95afbfee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:35.872303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249aba13-4022-4802-974c-cca95afbfee4 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:35.882183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 784/1559] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:35.883580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ce33c-64f6-4e0e-9d92-818cc4687fc0 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.883940 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ce33c-64f6-4e0e-9d92-818cc4687fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.884316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ce33c-64f6-4e0e-9d92-818cc4687fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.890198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 776/1560] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:35.891415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c491312-a4ec-4d0a-81da-92fe79eb1c5a None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.891706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c491312-a4ec-4d0a-81da-92fe79eb1c5a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.891985 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c491312-a4ec-4d0a-81da-92fe79eb1c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.907776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-7e3ce33c-64f6-4e0e-9d92-818cc4687fc0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.909017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7e3ce33c-64f6-4e0e-9d92-818cc4687fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.912658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-8c491312-a4ec-4d0a-81da-92fe79eb1c5a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.913875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8c491312-a4ec-4d0a-81da-92fe79eb1c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.917973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 785/1561] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:35.919114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6408a93-8f58-4bc2-83cc-faa4695608ef None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.919358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6408a93-8f58-4bc2-83cc-faa4695608ef None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.919635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6408a93-8f58-4bc2-83cc-faa4695608ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.923256 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 777/1562] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:35.925668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decac6a-cdf7-4c65-821d-7dd9b68dad64 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.925965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decac6a-cdf7-4c65-821d-7dd9b68dad64 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.926241 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decac6a-cdf7-4c65-821d-7dd9b68dad64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.937727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-b6408a93-8f58-4bc2-83cc-faa4695608ef None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.938921 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b6408a93-8f58-4bc2-83cc-faa4695608ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.946156 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-9decac6a-cdf7-4c65-821d-7dd9b68dad64 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.946731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 786/1563] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:35.950594 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-9decac6a-cdf7-4c65-821d-7dd9b68dad64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.950701 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6408a93-8f58-4bc2-83cc-faa4695608ef None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:35.956760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6408a93-8f58-4bc2-83cc-faa4695608ef None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:35.957574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 778/1564] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:35.958874 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52f987-7445-43e8-97b5-97453e1f6061 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.959167 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52f987-7445-43e8-97b5-97453e1f6061 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.959482 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52f987-7445-43e8-97b5-97453e1f6061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.964931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3ac40-a7b2-4f11-91e3-51cb5fe317cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:35.965581 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3ac40-a7b2-4f11-91e3-51cb5fe317cd admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.965866 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3ac40-a7b2-4f11-91e3-51cb5fe317cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.966147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3ac40-a7b2-4f11-91e3-51cb5fe317cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.966867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3ac40-a7b2-4f11-91e3-51cb5fe317cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:19:35.967204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3ac40-a7b2-4f11-91e3-51cb5fe317cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:35.968856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3ac40-a7b2-4f11-91e3-51cb5fe317cd admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:35.976667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-0b52f987-7445-43e8-97b5-97453e1f6061 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:35.977866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-0b52f987-7445-43e8-97b5-97453e1f6061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:35.983908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 787/1565] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:19:35.983908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3ac40-a7b2-4f11-91e3-51cb5fe317cd admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:35.992857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 779/1566] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:35.994010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d838be3-547e-4554-afb6-e21437de1cc0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:35.994941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d838be3-547e-4554-afb6-e21437de1cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:35.995225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d838be3-547e-4554-afb6-e21437de1cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:35.999778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3ac40-a7b2-4f11-91e3-51cb5fe317cd admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.004560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a949392-aef4-417b-a9fe-9ae777073344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.005211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a949392-aef4-417b-a9fe-9ae777073344 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.005494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a949392-aef4-417b-a9fe-9ae777073344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.005771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a949392-aef4-417b-a9fe-9ae777073344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.006484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a949392-aef4-417b-a9fe-9ae777073344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.008092 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a949392-aef4-417b-a9fe-9ae777073344 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.013153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-3d838be3-547e-4554-afb6-e21437de1cc0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:36.014412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3d838be3-547e-4554-afb6-e21437de1cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:36.019523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 788/1567] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:36.020343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a949392-aef4-417b-a9fe-9ae777073344 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.025812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 780/1568] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:36.025812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d838be3-547e-4554-afb6-e21437de1cc0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.029912 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a949392-aef4-417b-a9fe-9ae777073344 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.032479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d838be3-547e-4554-afb6-e21437de1cc0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.034357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9754b-2ba7-4464-975a-08b0477bd809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.034982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9754b-2ba7-4464-975a-08b0477bd809 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.035262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9754b-2ba7-4464-975a-08b0477bd809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.035540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9754b-2ba7-4464-975a-08b0477bd809 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.036539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9754b-2ba7-4464-975a-08b0477bd809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:19:36.036843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9754b-2ba7-4464-975a-08b0477bd809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.037377 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7480d-35e2-4d38-9dbd-5644d5a5299d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.038010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7480d-35e2-4d38-9dbd-5644d5a5299d admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.038292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7480d-35e2-4d38-9dbd-5644d5a5299d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.038407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9754b-2ba7-4464-975a-08b0477bd809 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.038565 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7480d-35e2-4d38-9dbd-5644d5a5299d admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.039378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7480d-35e2-4d38-9dbd-5644d5a5299d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:19:36.039854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7480d-35e2-4d38-9dbd-5644d5a5299d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.042437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7480d-35e2-4d38-9dbd-5644d5a5299d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.050107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 789/1569] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:19:36.050966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9754b-2ba7-4464-975a-08b0477bd809 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.062358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 781/1570] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:19:36.062358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7480d-35e2-4d38-9dbd-5644d5a5299d admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.062724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9754b-2ba7-4464-975a-08b0477bd809 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.066082 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605519e-895c-4899-b5f6-eaf94335c197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.066715 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605519e-895c-4899-b5f6-eaf94335c197 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.066999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605519e-895c-4899-b5f6-eaf94335c197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.067507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605519e-895c-4899-b5f6-eaf94335c197 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.068111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605519e-895c-4899-b5f6-eaf94335c197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:19:36.068443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605519e-895c-4899-b5f6-eaf94335c197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.070001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605519e-895c-4899-b5f6-eaf94335c197 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.072436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7480d-35e2-4d38-9dbd-5644d5a5299d admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.077133 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f3ec8-6ed0-46d3-a358-ad196a17defb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.077784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f3ec8-6ed0-46d3-a358-ad196a17defb admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.078076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f3ec8-6ed0-46d3-a358-ad196a17defb admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.078366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f3ec8-6ed0-46d3-a358-ad196a17defb admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.079089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f3ec8-6ed0-46d3-a358-ad196a17defb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:19:36.079463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f3ec8-6ed0-46d3-a358-ad196a17defb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.081080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f3ec8-6ed0-46d3-a358-ad196a17defb admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.081762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 790/1571] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:19:36.082564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605519e-895c-4899-b5f6-eaf94335c197 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.092789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605519e-895c-4899-b5f6-eaf94335c197 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.099465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 782/1572] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:19:36.099465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f3ec8-6ed0-46d3-a358-ad196a17defb admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.102507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a4f93-e14c-49fc-b0e7-0563ae1dec12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.103188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a4f93-e14c-49fc-b0e7-0563ae1dec12 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.104016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a4f93-e14c-49fc-b0e7-0563ae1dec12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.104587 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a4f93-e14c-49fc-b0e7-0563ae1dec12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.105772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a4f93-e14c-49fc-b0e7-0563ae1dec12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.107694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a4f93-e14c-49fc-b0e7-0563ae1dec12 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.138592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 791/1573] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:36.139678 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a4f93-e14c-49fc-b0e7-0563ae1dec12 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.153744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f3ec8-6ed0-46d3-a358-ad196a17defb admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.200324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a4f93-e14c-49fc-b0e7-0563ae1dec12 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.201478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60dd00-3e8b-48c9-be18-4e5fd80ec39b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.202131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60dd00-3e8b-48c9-be18-4e5fd80ec39b service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.202418 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60dd00-3e8b-48c9-be18-4e5fd80ec39b service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.202694 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60dd00-3e8b-48c9-be18-4e5fd80ec39b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.205342 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25498ea2-ab69-4494-99fa-702ad43c22ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.206339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25498ea2-ab69-4494-99fa-702ad43c22ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.206725 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25498ea2-ab69-4494-99fa-702ad43c22ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.207099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25498ea2-ab69-4494-99fa-702ad43c22ad admin admin] PATH_INFO: `/v3/projects/73cc2dce7dd1435ca89a36e3befc27aa/users/e81f4c0048484b7db56475814ad95caa/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.209634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:36.212415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60dd00-3e8b-48c9-be18-4e5fd80ec39b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.214863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60dd00-3e8b-48c9-be18-4e5fd80ec39b service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.222904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25498ea2-ab69-4494-99fa-702ad43c22ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=73cc2dce7dd1435ca89a36e3befc27aa, user_id=e81f4c0048484b7db56475814ad95caa, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.223909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: WARNING py.warnings [None req-25498ea2-ab69-4494-99fa-702ad43c22ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:19:36.223909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: warnings.warn(msg) Jun 17 07:19:36.223909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: Jun 17 07:19:36.225868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 783/1574] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:36.226007 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25498ea2-ab69-4494-99fa-702ad43c22ad admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.226545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60dd00-3e8b-48c9-be18-4e5fd80ec39b service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.238043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60dd00-3e8b-48c9-be18-4e5fd80ec39b service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.240376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 792/1575] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:36 2022] PUT /identity/v3/projects/73cc2dce7dd1435ca89a36e3befc27aa/users/e81f4c0048484b7db56475814ad95caa/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:36.241032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25498ea2-ab69-4494-99fa-702ad43c22ad admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.261947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:36.310117 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b57c3f-8aba-4026-a124-1d944c2a48ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.310987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b57c3f-8aba-4026-a124-1d944c2a48ae admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.311379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b57c3f-8aba-4026-a124-1d944c2a48ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.312384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b57c3f-8aba-4026-a124-1d944c2a48ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.313259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b57c3f-8aba-4026-a124-1d944c2a48ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:19:36.313665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b57c3f-8aba-4026-a124-1d944c2a48ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.315370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b57c3f-8aba-4026-a124-1d944c2a48ae admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.316895 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25498ea2-ab69-4494-99fa-702ad43c22ad admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.323373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d8386-4683-4f6c-8398-6f1ffe505f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.324166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d8386-4683-4f6c-8398-6f1ffe505f93 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.324955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d8386-4683-4f6c-8398-6f1ffe505f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.325261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d8386-4683-4f6c-8398-6f1ffe505f93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.325936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d8386-4683-4f6c-8398-6f1ffe505f93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.327983 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d8386-4683-4f6c-8398-6f1ffe505f93 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.328800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 784/1576] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:19:36.329867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b57c3f-8aba-4026-a124-1d944c2a48ae admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.342619 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b57c3f-8aba-4026-a124-1d944c2a48ae admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.348687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517104bf-16d9-4a50-99dd-d2ed3a8f3d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.349604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517104bf-16d9-4a50-99dd-d2ed3a8f3d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.349974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517104bf-16d9-4a50-99dd-d2ed3a8f3d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.350390 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517104bf-16d9-4a50-99dd-d2ed3a8f3d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.351125 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517104bf-16d9-4a50-99dd-d2ed3a8f3d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.353162 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517104bf-16d9-4a50-99dd-d2ed3a8f3d67 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.358722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 793/1577] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:36.359826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d8386-4683-4f6c-8398-6f1ffe505f93 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.380712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 785/1578] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:36.384100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517104bf-16d9-4a50-99dd-d2ed3a8f3d67 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.455796 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d8386-4683-4f6c-8398-6f1ffe505f93 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.473640 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517104bf-16d9-4a50-99dd-d2ed3a8f3d67 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.478883 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96e3bf-46a0-49fb-ae15-c56d637465d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.479557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96e3bf-46a0-49fb-ae15-c56d637465d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.479925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96e3bf-46a0-49fb-ae15-c56d637465d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.480148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96e3bf-46a0-49fb-ae15-c56d637465d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.480824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96e3bf-46a0-49fb-ae15-c56d637465d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.482736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96e3bf-46a0-49fb-ae15-c56d637465d3 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.521642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 786/1579] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:36.521869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874e267-80c2-45ca-8fd9-f4c652f6f3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.521990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96e3bf-46a0-49fb-ae15-c56d637465d3 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.522092 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874e267-80c2-45ca-8fd9-f4c652f6f3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.522092 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874e267-80c2-45ca-8fd9-f4c652f6f3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.522092 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874e267-80c2-45ca-8fd9-f4c652f6f3e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.522092 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874e267-80c2-45ca-8fd9-f4c652f6f3e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.523766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874e267-80c2-45ca-8fd9-f4c652f6f3e1 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.554751 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 794/1580] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/projects => generated 373 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:36.556688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e0fa3-823a-4228-a960-1b120e000a88 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.557130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e0fa3-823a-4228-a960-1b120e000a88 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.557631 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e0fa3-823a-4228-a960-1b120e000a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.583176 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:36.591375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-155e0fa3-823a-4228-a960-1b120e000a88 None None] MFA Rules not processed for user `e81f4c0048484b7db56475814ad95caa`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:36.593203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-155e0fa3-823a-4228-a960-1b120e000a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:36.595588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96e3bf-46a0-49fb-ae15-c56d637465d3 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.621480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:36.669625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734489dd-c7cc-4f27-922a-5cfc7be3bdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.670532 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734489dd-c7cc-4f27-922a-5cfc7be3bdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.670835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734489dd-c7cc-4f27-922a-5cfc7be3bdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.671116 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734489dd-c7cc-4f27-922a-5cfc7be3bdd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.671856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734489dd-c7cc-4f27-922a-5cfc7be3bdd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.674723 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734489dd-c7cc-4f27-922a-5cfc7be3bdd3 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.694365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 795/1581] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:36.695212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e0fa3-823a-4228-a960-1b120e000a88 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.702311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 787/1582] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:36.703122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734489dd-c7cc-4f27-922a-5cfc7be3bdd3 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.711585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e0fa3-823a-4228-a960-1b120e000a88 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.713842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734489dd-c7cc-4f27-922a-5cfc7be3bdd3 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.717442 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b875e5b-2b74-4ed2-aaf1-5ef0ac38b179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.718113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b875e5b-2b74-4ed2-aaf1-5ef0ac38b179 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.718394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d25d68-837f-4674-8f5e-ccc8991499bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.718863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b875e5b-2b74-4ed2-aaf1-5ef0ac38b179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.719057 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d25d68-837f-4674-8f5e-ccc8991499bf admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.719176 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b875e5b-2b74-4ed2-aaf1-5ef0ac38b179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.719335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d25d68-837f-4674-8f5e-ccc8991499bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.719725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d25d68-837f-4674-8f5e-ccc8991499bf admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.719929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b875e5b-2b74-4ed2-aaf1-5ef0ac38b179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.720434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d25d68-837f-4674-8f5e-ccc8991499bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.721856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b875e5b-2b74-4ed2-aaf1-5ef0ac38b179 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.722337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d25d68-837f-4674-8f5e-ccc8991499bf admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.746812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 788/1583] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/users => generated 287 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:36.746812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d25d68-837f-4674-8f5e-ccc8991499bf admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.757582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 796/1584] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:36.757582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b875e5b-2b74-4ed2-aaf1-5ef0ac38b179 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.758043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d25d68-837f-4674-8f5e-ccc8991499bf admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.826970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef565571-7d73-4011-b491-7ad06da6fc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.827904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef565571-7d73-4011-b491-7ad06da6fc25 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.828314 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef565571-7d73-4011-b491-7ad06da6fc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.828668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef565571-7d73-4011-b491-7ad06da6fc25 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.829918 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef565571-7d73-4011-b491-7ad06da6fc25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.832267 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef565571-7d73-4011-b491-7ad06da6fc25 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.832768 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b875e5b-2b74-4ed2-aaf1-5ef0ac38b179 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.837492 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1db82-8c13-49dd-87a7-99e21ed0383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.838572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1db82-8c13-49dd-87a7-99e21ed0383f admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.838572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1db82-8c13-49dd-87a7-99e21ed0383f admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.838953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1db82-8c13-49dd-87a7-99e21ed0383f admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.839704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1db82-8c13-49dd-87a7-99e21ed0383f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.842218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1db82-8c13-49dd-87a7-99e21ed0383f admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.857337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 789/1585] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:36.857337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef565571-7d73-4011-b491-7ad06da6fc25 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.868418 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 797/1586] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:36.868603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef565571-7d73-4011-b491-7ad06da6fc25 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.868740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1db82-8c13-49dd-87a7-99e21ed0383f admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.873388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b11ae-33a0-4f6d-9d80-3fb2cd61b95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.874233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b11ae-33a0-4f6d-9d80-3fb2cd61b95c admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.874522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b11ae-33a0-4f6d-9d80-3fb2cd61b95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.874807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b11ae-33a0-4f6d-9d80-3fb2cd61b95c admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.875567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b11ae-33a0-4f6d-9d80-3fb2cd61b95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.877611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b11ae-33a0-4f6d-9d80-3fb2cd61b95c admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.886551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1db82-8c13-49dd-87a7-99e21ed0383f admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.893925 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddc95f-6d76-4a0e-9878-40b4049d7e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.894845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddc95f-6d76-4a0e-9878-40b4049d7e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.895335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddc95f-6d76-4a0e-9878-40b4049d7e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.895770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddc95f-6d76-4a0e-9878-40b4049d7e67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.896703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebddc95f-6d76-4a0e-9878-40b4049d7e67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.898908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebddc95f-6d76-4a0e-9878-40b4049d7e67 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.902250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 790/1587] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:36.903170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b11ae-33a0-4f6d-9d80-3fb2cd61b95c admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.913513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b11ae-33a0-4f6d-9d80-3fb2cd61b95c admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.919087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0098574-2a7b-4ef1-9386-3a4232a86d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:36.919868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0098574-2a7b-4ef1-9386-3a4232a86d48 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:36.920167 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0098574-2a7b-4ef1-9386-3a4232a86d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:36.920454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0098574-2a7b-4ef1-9386-3a4232a86d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:36.921234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0098574-2a7b-4ef1-9386-3a4232a86d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:36.923049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0098574-2a7b-4ef1-9386-3a4232a86d48 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:36.937236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 791/1588] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:36.940860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0098574-2a7b-4ef1-9386-3a4232a86d48 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.952202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0098574-2a7b-4ef1-9386-3a4232a86d48 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:36.956242 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 798/1589] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/projects => generated 405 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:36.957637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddc95f-6d76-4a0e-9878-40b4049d7e67 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:36.985112 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:37.016681 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9316e0e-0f09-4362-ac93-60e345566cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.017672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9316e0e-0f09-4362-ac93-60e345566cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.018064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9316e0e-0f09-4362-ac93-60e345566cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.018463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9316e0e-0f09-4362-ac93-60e345566cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.019309 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9316e0e-0f09-4362-ac93-60e345566cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.021167 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9316e0e-0f09-4362-ac93-60e345566cf8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.034813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddc95f-6d76-4a0e-9878-40b4049d7e67 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.035399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 792/1590] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:37.036672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9316e0e-0f09-4362-ac93-60e345566cf8 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.041140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba749ef-aecb-46d4-bdac-d77217603a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.041719 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba749ef-aecb-46d4-bdac-d77217603a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.042004 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba749ef-aecb-46d4-bdac-d77217603a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.042314 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba749ef-aecb-46d4-bdac-d77217603a13 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.043376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba749ef-aecb-46d4-bdac-d77217603a13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.045864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba749ef-aecb-46d4-bdac-d77217603a13 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.047998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9316e0e-0f09-4362-ac93-60e345566cf8 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.053124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e31e77-db74-4008-9f0c-66df500ec630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.053875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31e77-db74-4008-9f0c-66df500ec630 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.054239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31e77-db74-4008-9f0c-66df500ec630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.054606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31e77-db74-4008-9f0c-66df500ec630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.055405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e31e77-db74-4008-9f0c-66df500ec630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.058092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e31e77-db74-4008-9f0c-66df500ec630 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.065138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:37.070076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 793/1591] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:37.070941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e31e77-db74-4008-9f0c-66df500ec630 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.076519 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 799/1592] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:36 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:37.077507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba749ef-aecb-46d4-bdac-d77217603a13 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.087207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e31e77-db74-4008-9f0c-66df500ec630 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.091260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba749ef-aecb-46d4-bdac-d77217603a13 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.092304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb4cb0-a0a9-49bf-a06d-505d1561813f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.092980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb4cb0-a0a9-49bf-a06d-505d1561813f admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.093268 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb4cb0-a0a9-49bf-a06d-505d1561813f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.093551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb4cb0-a0a9-49bf-a06d-505d1561813f admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.094273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb4cb0-a0a9-49bf-a06d-505d1561813f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.095892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb4cb0-a0a9-49bf-a06d-505d1561813f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.096972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233558b-e6de-4757-8de5-72907fa61598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.097944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233558b-e6de-4757-8de5-72907fa61598 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.098707 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233558b-e6de-4757-8de5-72907fa61598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.099095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233558b-e6de-4757-8de5-72907fa61598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.100119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233558b-e6de-4757-8de5-72907fa61598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.102285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233558b-e6de-4757-8de5-72907fa61598 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.109944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 794/1593] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:37.109944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb4cb0-a0a9-49bf-a06d-505d1561813f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.114446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 800/1594] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:37.115562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233558b-e6de-4757-8de5-72907fa61598 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.122330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb4cb0-a0a9-49bf-a06d-505d1561813f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.127780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39209f6-53f2-46df-be34-f58c4f3009c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.128512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39209f6-53f2-46df-be34-f58c4f3009c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.128802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39209f6-53f2-46df-be34-f58c4f3009c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.129132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39209f6-53f2-46df-be34-f58c4f3009c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.129865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39209f6-53f2-46df-be34-f58c4f3009c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.130293 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233558b-e6de-4757-8de5-72907fa61598 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.131665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39209f6-53f2-46df-be34-f58c4f3009c4 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.139129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b61823-5ed0-4377-8f34-f2795688aaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.140283 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b61823-5ed0-4377-8f34-f2795688aaa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.140676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b61823-5ed0-4377-8f34-f2795688aaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.141056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b61823-5ed0-4377-8f34-f2795688aaa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.141888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b61823-5ed0-4377-8f34-f2795688aaa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.143126 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 795/1595] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:37.144868 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b61823-5ed0-4377-8f34-f2795688aaa7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.147116 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39209f6-53f2-46df-be34-f58c4f3009c4 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.157537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39209f6-53f2-46df-be34-f58c4f3009c4 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.162868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817477b-d608-43b9-bf71-fca6f426dfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.163547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817477b-d608-43b9-bf71-fca6f426dfc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.163833 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817477b-d608-43b9-bf71-fca6f426dfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.164161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817477b-d608-43b9-bf71-fca6f426dfc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.164885 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817477b-d608-43b9-bf71-fca6f426dfc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.166535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817477b-d608-43b9-bf71-fca6f426dfc7 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.175551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 801/1596] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:37 2022] POST /identity/v3/users => generated 309 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:37.175551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b61823-5ed0-4377-8f34-f2795688aaa7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.184020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 796/1597] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:37.184952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817477b-d608-43b9-bf71-fca6f426dfc7 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.186503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b61823-5ed0-4377-8f34-f2795688aaa7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.191352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0483b-450e-4fcc-9b91-24a7fe2ec2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.192471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0483b-450e-4fcc-9b91-24a7fe2ec2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.192889 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0483b-450e-4fcc-9b91-24a7fe2ec2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.193250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0483b-450e-4fcc-9b91-24a7fe2ec2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.194136 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0483b-450e-4fcc-9b91-24a7fe2ec2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.195675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817477b-d608-43b9-bf71-fca6f426dfc7 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.195941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0483b-450e-4fcc-9b91-24a7fe2ec2d7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.202761 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70e984-1012-4698-b433-1bb017cb95c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.203622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70e984-1012-4698-b433-1bb017cb95c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.204604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70e984-1012-4698-b433-1bb017cb95c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.205036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70e984-1012-4698-b433-1bb017cb95c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.206064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70e984-1012-4698-b433-1bb017cb95c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.208232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70e984-1012-4698-b433-1bb017cb95c0 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.214542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 802/1598] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:37.216108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0483b-450e-4fcc-9b91-24a7fe2ec2d7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.223764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 797/1599] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:37.225068 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70e984-1012-4698-b433-1bb017cb95c0 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.231364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0483b-450e-4fcc-9b91-24a7fe2ec2d7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.242099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70e984-1012-4698-b433-1bb017cb95c0 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.243269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f2f26-ae58-46c5-a766-d123f4550dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.244739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f2f26-ae58-46c5-a766-d123f4550dac admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.245554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f2f26-ae58-46c5-a766-d123f4550dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.245851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f2f26-ae58-46c5-a766-d123f4550dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.246573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f2f26-ae58-46c5-a766-d123f4550dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.248520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f2f26-ae58-46c5-a766-d123f4550dac admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.253672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1859be1-666d-41f0-a07e-c42afb8a32eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.254568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1859be1-666d-41f0-a07e-c42afb8a32eb admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.255615 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1859be1-666d-41f0-a07e-c42afb8a32eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.256093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1859be1-666d-41f0-a07e-c42afb8a32eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.257081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1859be1-666d-41f0-a07e-c42afb8a32eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.259286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1859be1-666d-41f0-a07e-c42afb8a32eb admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.263648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 803/1600] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:37.263648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f2f26-ae58-46c5-a766-d123f4550dac admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.277025 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 798/1601] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:37.278116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f2f26-ae58-46c5-a766-d123f4550dac admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.278612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1859be1-666d-41f0-a07e-c42afb8a32eb admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.285042 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1b206-a938-4e0f-b1aa-a5b149268aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.286767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1b206-a938-4e0f-b1aa-a5b149268aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.287163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1b206-a938-4e0f-b1aa-a5b149268aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.287547 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1b206-a938-4e0f-b1aa-a5b149268aaf admin admin] PATH_INFO: `/v3/projects/2b581a57e3e44e3481b1db445961afc5/users/c11f44eb67cd4923a840171833bb572a/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.293208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1859be1-666d-41f0-a07e-c42afb8a32eb admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.299393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a606c4-341e-40bd-9b03-53eb1eac44de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.300179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a606c4-341e-40bd-9b03-53eb1eac44de admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.300518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a606c4-341e-40bd-9b03-53eb1eac44de admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.300998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a606c4-341e-40bd-9b03-53eb1eac44de admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.301959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a606c4-341e-40bd-9b03-53eb1eac44de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.303676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a606c4-341e-40bd-9b03-53eb1eac44de admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.308931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1b206-a938-4e0f-b1aa-a5b149268aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b581a57e3e44e3481b1db445961afc5, user_id=c11f44eb67cd4923a840171833bb572a, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.312387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1b206-a938-4e0f-b1aa-a5b149268aaf admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.316514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 799/1602] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:37.321330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a606c4-341e-40bd-9b03-53eb1eac44de admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.330706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 804/1603] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:37 2022] PUT /identity/v3/projects/2b581a57e3e44e3481b1db445961afc5/users/c11f44eb67cd4923a840171833bb572a/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:37.332408 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1b206-a938-4e0f-b1aa-a5b149268aaf admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.339062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a606c4-341e-40bd-9b03-53eb1eac44de admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.409806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dc360-195d-473b-b1ef-58d588c9007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.410785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dc360-195d-473b-b1ef-58d588c9007f admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.411495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dc360-195d-473b-b1ef-58d588c9007f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.411783 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dc360-195d-473b-b1ef-58d588c9007f admin admin] PATH_INFO: `/v3/projects/ca5bba369109460b8a44d927072bd277/users/3d519a07e8434aaa95adb641b669838f/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.435390 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dc360-195d-473b-b1ef-58d588c9007f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5bba369109460b8a44d927072bd277, user_id=3d519a07e8434aaa95adb641b669838f, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.438452 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dc360-195d-473b-b1ef-58d588c9007f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.439838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1b206-a938-4e0f-b1aa-a5b149268aaf admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.446306 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42228e3a-6808-434a-8ee1-393ef1306455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.447689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42228e3a-6808-434a-8ee1-393ef1306455 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.448690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42228e3a-6808-434a-8ee1-393ef1306455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.449322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42228e3a-6808-434a-8ee1-393ef1306455 admin admin] PATH_INFO: `/v3/projects/277f81ef9ed74a6189221440edd53d4c/users/31e94cc1c6a743ce898424ad7ba26cf1/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.464653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 800/1604] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:37 2022] PUT /identity/v3/projects/ca5bba369109460b8a44d927072bd277/users/3d519a07e8434aaa95adb641b669838f/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:37.466033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dc360-195d-473b-b1ef-58d588c9007f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.471481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42228e3a-6808-434a-8ee1-393ef1306455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277f81ef9ed74a6189221440edd53d4c, user_id=31e94cc1c6a743ce898424ad7ba26cf1, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.476979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42228e3a-6808-434a-8ee1-393ef1306455 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.497945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 805/1605] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:37 2022] PUT /identity/v3/projects/277f81ef9ed74a6189221440edd53d4c/users/31e94cc1c6a743ce898424ad7ba26cf1/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:37.499155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42228e3a-6808-434a-8ee1-393ef1306455 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.527894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:37.564096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dc360-195d-473b-b1ef-58d588c9007f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.581453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42228e3a-6808-434a-8ee1-393ef1306455 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.589026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99487b55-9e34-41b9-8bec-1d091342fd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.590016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99487b55-9e34-41b9-8bec-1d091342fd6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.590335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99487b55-9e34-41b9-8bec-1d091342fd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.590717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99487b55-9e34-41b9-8bec-1d091342fd6a admin admin] PATH_INFO: `/v3/projects/09dfc5c0d3f4405a901acd8eeef080b9/users/6d04750a83fb465e8fba24c1143d4eff/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.604344 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:37.627386 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99487b55-9e34-41b9-8bec-1d091342fd6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=09dfc5c0d3f4405a901acd8eeef080b9, user_id=6d04750a83fb465e8fba24c1143d4eff, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.631409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99487b55-9e34-41b9-8bec-1d091342fd6a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.649433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 806/1606] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:37 2022] PUT /identity/v3/projects/09dfc5c0d3f4405a901acd8eeef080b9/users/6d04750a83fb465e8fba24c1143d4eff/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:37.651090 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99487b55-9e34-41b9-8bec-1d091342fd6a admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.653791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a4475-521a-466b-a9aa-297af67456d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.656408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a4475-521a-466b-a9aa-297af67456d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.656775 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a4475-521a-466b-a9aa-297af67456d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.657231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a4475-521a-466b-a9aa-297af67456d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.657972 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a4475-521a-466b-a9aa-297af67456d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.661119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a4475-521a-466b-a9aa-297af67456d3 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.677948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 801/1607] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:37.679132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a4475-521a-466b-a9aa-297af67456d3 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:37.691526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a4475-521a-466b-a9aa-297af67456d3 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.729633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99487b55-9e34-41b9-8bec-1d091342fd6a admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:37.737183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6dcb68-6445-4ffe-a212-dd196792e1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.738150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dcb68-6445-4ffe-a212-dd196792e1d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.738373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dcb68-6445-4ffe-a212-dd196792e1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.738653 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dcb68-6445-4ffe-a212-dd196792e1d0 admin admin] PATH_INFO: `/v3/projects/7bdb0b874c434af39fb644908137de77/users/ad2d4ce6c4b840b78abc19c28063a747/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.758202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6dcb68-6445-4ffe-a212-dd196792e1d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bdb0b874c434af39fb644908137de77, user_id=ad2d4ce6c4b840b78abc19c28063a747, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.760957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6dcb68-6445-4ffe-a212-dd196792e1d0 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.765497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f0d7b-be69-44d8-9014-88cd3e1a7d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:37.766648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f0d7b-be69-44d8-9014-88cd3e1a7d09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.767130 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f0d7b-be69-44d8-9014-88cd3e1a7d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.767415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f0d7b-be69-44d8-9014-88cd3e1a7d09 admin admin] PATH_INFO: `/v3/projects/96d2a4641fe84ed396e4cd1acb17ab34/users/bbce34136ede42029c45a6b3c89b77b0/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.782649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 807/1608] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:37 2022] PUT /identity/v3/projects/7bdb0b874c434af39fb644908137de77/users/ad2d4ce6c4b840b78abc19c28063a747/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:37.784184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eba43a-5b58-4b09-b0fb-aa765b6710b5 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.784535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eba43a-5b58-4b09-b0fb-aa765b6710b5 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.784882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eba43a-5b58-4b09-b0fb-aa765b6710b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.797065 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f0d7b-be69-44d8-9014-88cd3e1a7d09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96d2a4641fe84ed396e4cd1acb17ab34, user_id=bbce34136ede42029c45a6b3c89b77b0, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:37.799781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f0d7b-be69-44d8-9014-88cd3e1a7d09 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:37.806058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c3eba43a-5b58-4b09-b0fb-aa765b6710b5 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:37.807498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c3eba43a-5b58-4b09-b0fb-aa765b6710b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:37.821038 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 802/1609] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:37 2022] PUT /identity/v3/projects/96d2a4641fe84ed396e4cd1acb17ab34/users/bbce34136ede42029c45a6b3c89b77b0/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:37.822591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ac621-39f4-46a1-8aff-4139715757f5 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.822963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ac621-39f4-46a1-8aff-4139715757f5 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.823327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ac621-39f4-46a1-8aff-4139715757f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.863683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-ca1ac621-39f4-46a1-8aff-4139715757f5 None None] MFA Rules not processed for user `3d519a07e8434aaa95adb641b669838f`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:37.864527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ca1ac621-39f4-46a1-8aff-4139715757f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:37.913538 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 808/1610] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:37.914784 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472cca2-d038-4ff2-a670-58fce293ae99 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.915183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472cca2-d038-4ff2-a670-58fce293ae99 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.915574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472cca2-d038-4ff2-a670-58fce293ae99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.939715 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-e472cca2-d038-4ff2-a670-58fce293ae99 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:37.941123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e472cca2-d038-4ff2-a670-58fce293ae99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:37.941700 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:37.970469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 803/1611] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:37.970469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0a369-af0d-4fd5-9e8d-4957e361dc17 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:37.970469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0a369-af0d-4fd5-9e8d-4957e361dc17 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:37.970469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0a369-af0d-4fd5-9e8d-4957e361dc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:37.993430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-acb0a369-af0d-4fd5-9e8d-4957e361dc17 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:37.994920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-acb0a369-af0d-4fd5-9e8d-4957e361dc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:38.044213 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 809/1612] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:38.045124 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472cca2-d038-4ff2-a670-58fce293ae99 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:38.054738 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472cca2-d038-4ff2-a670-58fce293ae99 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:38.059524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df56a2-7382-4730-9c80-840657f8637a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:38.060478 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df56a2-7382-4730-9c80-840657f8637a admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.060761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df56a2-7382-4730-9c80-840657f8637a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.061044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df56a2-7382-4730-9c80-840657f8637a admin admin] PATH_INFO: `/v3/projects/5ab9e078b5da4de285e3b82cf3d79e08/users/4336f95f0b0c483e92db5a8ac058c9c1/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.075483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:38.081648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df56a2-7382-4730-9c80-840657f8637a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab9e078b5da4de285e3b82cf3d79e08, user_id=4336f95f0b0c483e92db5a8ac058c9c1, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:38.085058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df56a2-7382-4730-9c80-840657f8637a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:38.102938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 810/1613] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:38 2022] PUT /identity/v3/projects/5ab9e078b5da4de285e3b82cf3d79e08/users/4336f95f0b0c483e92db5a8ac058c9c1/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:38.104448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d798207-5c5e-475c-af66-1ef2b8683720 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.104848 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d798207-5c5e-475c-af66-1ef2b8683720 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.105252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d798207-5c5e-475c-af66-1ef2b8683720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.209284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 804/1614] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:38.209284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0a369-af0d-4fd5-9e8d-4957e361dc17 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:38.209284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0a369-af0d-4fd5-9e8d-4957e361dc17 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:38.210100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3d798207-5c5e-475c-af66-1ef2b8683720 None None] MFA Rules not processed for user `ad2d4ce6c4b840b78abc19c28063a747`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:38.210100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3d798207-5c5e-475c-af66-1ef2b8683720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:38.254114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af889197-3923-487c-ab92-9070a7b89371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:38.256277 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af889197-3923-487c-ab92-9070a7b89371 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.256724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af889197-3923-487c-ab92-9070a7b89371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.257137 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af889197-3923-487c-ab92-9070a7b89371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.257889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af889197-3923-487c-ab92-9070a7b89371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:38.260053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af889197-3923-487c-ab92-9070a7b89371 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:38.272001 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 805/1615] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:38.272914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af889197-3923-487c-ab92-9070a7b89371 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:38.273076 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 811/1616] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:38.274643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-af889197-3923-487c-ab92-9070a7b89371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:38.274773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd171eac-41d9-42aa-87dc-bb9b71844c1a None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.275054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd171eac-41d9-42aa-87dc-bb9b71844c1a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.276168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd171eac-41d9-42aa-87dc-bb9b71844c1a None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.277903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 812/1617] 10.222.0.33 () {54 vars in 961 bytes} [Fri Jun 17 07:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:19:38.278375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd171eac-41d9-42aa-87dc-bb9b71844c1a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:38.285019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-cd171eac-41d9-42aa-87dc-bb9b71844c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:38.358374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd171eac-41d9-42aa-87dc-bb9b71844c1a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:38.361922 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da190b-c2eb-47c9-a1e0-6b80805f20d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:38.362674 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da190b-c2eb-47c9-a1e0-6b80805f20d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.362962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da190b-c2eb-47c9-a1e0-6b80805f20d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.363235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da190b-c2eb-47c9-a1e0-6b80805f20d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.368801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af889197-3923-487c-ab92-9070a7b89371 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:38.373836 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da190b-c2eb-47c9-a1e0-6b80805f20d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:38.374760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df60e90-a8ed-4f6d-9661-723a99005fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:38.375201 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da190b-c2eb-47c9-a1e0-6b80805f20d0 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:38.375581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df60e90-a8ed-4f6d-9661-723a99005fad service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.375938 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df60e90-a8ed-4f6d-9661-723a99005fad service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.376808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df60e90-a8ed-4f6d-9661-723a99005fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.386541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 813/1618] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:38.392971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df60e90-a8ed-4f6d-9661-723a99005fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:38.394730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df60e90-a8ed-4f6d-9661-723a99005fad service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:38.397653 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fa626-4be4-4796-87e9-9c7ee87a6e00 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.398129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fa626-4be4-4796-87e9-9c7ee87a6e00 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.398571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fa626-4be4-4796-87e9-9c7ee87a6e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.406630 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 806/1619] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:38.407116 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df60e90-a8ed-4f6d-9661-723a99005fad service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:38.417789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df60e90-a8ed-4f6d-9661-723a99005fad service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:38.422831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d0c73-d708-45fe-a905-b9243e75c552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:38.423498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d0c73-d708-45fe-a905-b9243e75c552 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.423785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d0c73-d708-45fe-a905-b9243e75c552 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.424085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d0c73-d708-45fe-a905-b9243e75c552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.430677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-601fa626-4be4-4796-87e9-9c7ee87a6e00 None None] MFA Rules not processed for user `4336f95f0b0c483e92db5a8ac058c9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:38.432655 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-601fa626-4be4-4796-87e9-9c7ee87a6e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:38.440758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d0c73-d708-45fe-a905-b9243e75c552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:38.442304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d0c73-d708-45fe-a905-b9243e75c552 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:38.454014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 807/1620] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:38.454848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d0c73-d708-45fe-a905-b9243e75c552 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:38.465324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d0c73-d708-45fe-a905-b9243e75c552 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:38.473477 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eb58c-e699-4d40-af96-8ec2d4707d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:38.474406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eb58c-e699-4d40-af96-8ec2d4707d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.474859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eb58c-e699-4d40-af96-8ec2d4707d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.475000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eb58c-e699-4d40-af96-8ec2d4707d6e admin admin] PATH_INFO: `/v3/projects/96d2a4641fe84ed396e4cd1acb17ab34/users/bbce34136ede42029c45a6b3c89b77b0/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.477977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86eb58c-e699-4d40-af96-8ec2d4707d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=96d2a4641fe84ed396e4cd1acb17ab34, user_id=bbce34136ede42029c45a6b3c89b77b0, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:38.481637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86eb58c-e699-4d40-af96-8ec2d4707d6e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:38.498545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 808/1621] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:38 2022] PUT /identity/v3/projects/96d2a4641fe84ed396e4cd1acb17ab34/users/bbce34136ede42029c45a6b3c89b77b0/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:38.510933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688d335-ee41-4bde-a0cc-5c7e6124de4f None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.511230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688d335-ee41-4bde-a0cc-5c7e6124de4f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.511935 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688d335-ee41-4bde-a0cc-5c7e6124de4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.516220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 814/1622] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:38.517740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05539109-7027-4616-b1d0-b1726138aef1 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.518033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05539109-7027-4616-b1d0-b1726138aef1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.519251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05539109-7027-4616-b1d0-b1726138aef1 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.520794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 815/1623] 10.222.0.33 () {54 vars in 961 bytes} [Fri Jun 17 07:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:19:38.523123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdda222-078b-423b-8ad5-2c411c24e06c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.523416 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdda222-078b-423b-8ad5-2c411c24e06c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.523755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdda222-078b-423b-8ad5-2c411c24e06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.529600 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:38.552330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-b688d335-ee41-4bde-a0cc-5c7e6124de4f None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:38.553454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b688d335-ee41-4bde-a0cc-5c7e6124de4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:38.567689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:38.581116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-5fdda222-078b-423b-8ad5-2c411c24e06c None None] MFA Rules not processed for user `bbce34136ede42029c45a6b3c89b77b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:38.582514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-5fdda222-078b-423b-8ad5-2c411c24e06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:38.670146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 809/1624] 10.222.0.33 () {58 vars in 1058 bytes} [Fri Jun 17 07:19:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:38.671083 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688d335-ee41-4bde-a0cc-5c7e6124de4f None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:38.682431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 816/1625] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:38 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:38.683969 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d36e94-b947-422d-a2b6-a6267d697481 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.684885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d36e94-b947-422d-a2b6-a6267d697481 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.684885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d36e94-b947-422d-a2b6-a6267d697481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.722532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-30d36e94-b947-422d-a2b6-a6267d697481 None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:38.725378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-30d36e94-b947-422d-a2b6-a6267d697481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:38.747571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 817/1626] 10.222.0.33 () {58 vars in 1058 bytes} [Fri Jun 17 07:19:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:38.749153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d36e94-b947-422d-a2b6-a6267d697481 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:38.764797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d36e94-b947-422d-a2b6-a6267d697481 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:38.774181 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909044eb-1d7b-4d0e-8990-9fa28459e588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:38.775592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909044eb-1d7b-4d0e-8990-9fa28459e588 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.776618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909044eb-1d7b-4d0e-8990-9fa28459e588 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.777227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909044eb-1d7b-4d0e-8990-9fa28459e588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.813970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688d335-ee41-4bde-a0cc-5c7e6124de4f None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:38.825473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b2547-d501-44f2-89cf-11efa701a23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:38.826479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b2547-d501-44f2-89cf-11efa701a23f admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:38.826721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b2547-d501-44f2-89cf-11efa701a23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:38.827622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b2547-d501-44f2-89cf-11efa701a23f admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:38.828368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b2547-d501-44f2-89cf-11efa701a23f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:38.831060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b2547-d501-44f2-89cf-11efa701a23f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:38.876257 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909044eb-1d7b-4d0e-8990-9fa28459e588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:38.879301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909044eb-1d7b-4d0e-8990-9fa28459e588 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:38.880978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 810/1627] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:38 2022] POST /identity/v3/projects => generated 375 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:38.882460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b2547-d501-44f2-89cf-11efa701a23f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:38.907348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 818/1628] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:38.908387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909044eb-1d7b-4d0e-8990-9fa28459e588 service nova] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:38.936867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909044eb-1d7b-4d0e-8990-9fa28459e588 service nova] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:39.006428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b2547-d501-44f2-89cf-11efa701a23f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:39.012434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967068c8-a703-403d-b6b4-dc4a85c36b92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:39.013811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967068c8-a703-403d-b6b4-dc4a85c36b92 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:39.014295 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967068c8-a703-403d-b6b4-dc4a85c36b92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:39.015212 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967068c8-a703-403d-b6b4-dc4a85c36b92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:39.083825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddecba-24e8-4048-a051-3928b641f495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:39.084767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddecba-24e8-4048-a051-3928b641f495 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:39.085161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddecba-24e8-4048-a051-3928b641f495 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:39.085690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddecba-24e8-4048-a051-3928b641f495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:39.110824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:39.151623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967068c8-a703-403d-b6b4-dc4a85c36b92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:39.154707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967068c8-a703-403d-b6b4-dc4a85c36b92 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:39.192341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 811/1629] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:38 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:39.194450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967068c8-a703-403d-b6b4-dc4a85c36b92 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:39.369176 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967068c8-a703-403d-b6b4-dc4a85c36b92 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:39.380716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25112db-a292-4781-97ce-74fab903b72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:39.381431 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25112db-a292-4781-97ce-74fab903b72c admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:39.381720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25112db-a292-4781-97ce-74fab903b72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:39.382002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25112db-a292-4781-97ce-74fab903b72c admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:39.382731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25112db-a292-4781-97ce-74fab903b72c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:39.386883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:39.409007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25112db-a292-4781-97ce-74fab903b72c admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:39.471724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddecba-24e8-4048-a051-3928b641f495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:39.473393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddecba-24e8-4048-a051-3928b641f495 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:39.480546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 812/1630] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:39 2022] POST /identity/v3/users => generated 294 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:39.484345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25112db-a292-4781-97ce-74fab903b72c admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:39.508105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 819/1631] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:39.510027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0c012-e492-4d87-aedc-ffdad764539c None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:39.510332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0c012-e492-4d87-aedc-ffdad764539c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:39.510610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0c012-e492-4d87-aedc-ffdad764539c None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:39.512030 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 820/1632] 10.222.0.33 () {54 vars in 912 bytes} [Fri Jun 17 07:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:19:39.521996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25112db-a292-4781-97ce-74fab903b72c admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:39.525739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29676e5-ab41-46e2-bd20-96e25df5d50c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:39.526033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29676e5-ab41-46e2-bd20-96e25df5d50c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:39.526312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29676e5-ab41-46e2-bd20-96e25df5d50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:39.553405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b81a1-2369-4f2d-b121-0d0e3f40c383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:39.554143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b81a1-2369-4f2d-b121-0d0e3f40c383 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:39.554431 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b81a1-2369-4f2d-b121-0d0e3f40c383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:39.554711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b81a1-2369-4f2d-b121-0d0e3f40c383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:39.555455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b81a1-2369-4f2d-b121-0d0e3f40c383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:39.566819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b81a1-2369-4f2d-b121-0d0e3f40c383 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:39.566949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-d29676e5-ab41-46e2-bd20-96e25df5d50c None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:39.568260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d29676e5-ab41-46e2-bd20-96e25df5d50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:39.592840 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 821/1633] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:19:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:39.593806 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29676e5-ab41-46e2-bd20-96e25df5d50c None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:39.617390 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 813/1634] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:39.619740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29676e5-ab41-46e2-bd20-96e25df5d50c None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:39.622508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b81a1-2369-4f2d-b121-0d0e3f40c383 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:39.633258 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d32c7a-85d8-4862-9367-5a9efcf02aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:39.634152 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d32c7a-85d8-4862-9367-5a9efcf02aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:39.634435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d32c7a-85d8-4862-9367-5a9efcf02aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:39.635244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d32c7a-85d8-4862-9367-5a9efcf02aae admin admin] PATH_INFO: `/v3/users/e81f4c0048484b7db56475814ad95caa` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:39.642302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d32c7a-85d8-4862-9367-5a9efcf02aae admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81f4c0048484b7db56475814ad95caa)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:39.643361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: WARNING py.warnings [None req-52d32c7a-85d8-4862-9367-5a9efcf02aae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:19:39.643361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: warnings.warn(msg) Jun 17 07:19:39.643361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: Jun 17 07:19:39.644622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d32c7a-85d8-4862-9367-5a9efcf02aae admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:39.661129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b81a1-2369-4f2d-b121-0d0e3f40c383 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:39.677355 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f487f-2afb-46f8-a669-e75581588c7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:39.677807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f487f-2afb-46f8-a669-e75581588c7f service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:39.678099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f487f-2afb-46f8-a669-e75581588c7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:39.678380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f487f-2afb-46f8-a669-e75581588c7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:39.716616 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f487f-2afb-46f8-a669-e75581588c7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:39.725433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f487f-2afb-46f8-a669-e75581588c7f service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:39.754112 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 814/1635] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:39.754948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f487f-2afb-46f8-a669-e75581588c7f service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:39.786767 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f487f-2afb-46f8-a669-e75581588c7f service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:39.794664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-52d32c7a-85d8-4862-9367-5a9efcf02aae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81f4c0048484b7db56475814ad95caa'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:39.795580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-52d32c7a-85d8-4862-9367-5a9efcf02aae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81f4c0048484b7db56475814ad95caa'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:39.798799 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc7aae-76b1-420c-8eeb-035b38d6ff4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:39.799714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc7aae-76b1-420c-8eeb-035b38d6ff4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:39.799997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc7aae-76b1-420c-8eeb-035b38d6ff4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:39.803394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc7aae-76b1-420c-8eeb-035b38d6ff4f admin admin] PATH_INFO: `/v3/projects/0de8e4142bed424d8845afcd9dbeaf01/users/b5c29d7121a44c3f85abc61d1eda8320/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:39.826466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-52d32c7a-85d8-4862-9367-5a9efcf02aae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81f4c0048484b7db56475814ad95caa'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:39.844911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc7aae-76b1-420c-8eeb-035b38d6ff4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de8e4142bed424d8845afcd9dbeaf01, user_id=b5c29d7121a44c3f85abc61d1eda8320, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:39.853501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc7aae-76b1-420c-8eeb-035b38d6ff4f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:39.880701 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 815/1636] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:39 2022] PUT /identity/v3/projects/0de8e4142bed424d8845afcd9dbeaf01/users/b5c29d7121a44c3f85abc61d1eda8320/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:39.885588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc7aae-76b1-420c-8eeb-035b38d6ff4f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:39.893758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-66fc7aae-76b1-420c-8eeb-035b38d6ff4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:39.901087 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-52d32c7a-85d8-4862-9367-5a9efcf02aae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81f4c0048484b7db56475814ad95caa'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:39.905399 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-52d32c7a-85d8-4862-9367-5a9efcf02aae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81f4c0048484b7db56475814ad95caa'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:39.931975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 822/1637] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:19:39 2022] DELETE /identity/v3/users/e81f4c0048484b7db56475814ad95caa => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:39.936666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07a6ca-e4b2-49d5-af17-c56bf21679d5 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:39.937337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07a6ca-e4b2-49d5-af17-c56bf21679d5 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:39.937800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07a6ca-e4b2-49d5-af17-c56bf21679d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:39.992787 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-1d07a6ca-e4b2-49d5-af17-c56bf21679d5 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:39.995846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-1d07a6ca-e4b2-49d5-af17-c56bf21679d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:40.059867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc7aae-76b1-420c-8eeb-035b38d6ff4f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:40.113489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:40.213733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 823/1638] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:40.232190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41a9f2-b47f-4b64-befe-cd7dfaea0d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:40.235461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41a9f2-b47f-4b64-befe-cd7dfaea0d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:40.235749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41a9f2-b47f-4b64-befe-cd7dfaea0d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:40.236049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41a9f2-b47f-4b64-befe-cd7dfaea0d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:40.239017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41a9f2-b47f-4b64-befe-cd7dfaea0d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:40.243289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41a9f2-b47f-4b64-befe-cd7dfaea0d68 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:40.275665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 816/1639] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:40.279945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07a6ca-e4b2-49d5-af17-c56bf21679d5 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:40.291728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07a6ca-e4b2-49d5-af17-c56bf21679d5 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:40.296970 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf2231-7f77-41cc-9376-744a51ae664e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:40.298075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf2231-7f77-41cc-9376-744a51ae664e admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:40.298567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf2231-7f77-41cc-9376-744a51ae664e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:40.299011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf2231-7f77-41cc-9376-744a51ae664e admin admin] PATH_INFO: `/v3/projects/0de8e4142bed424d8845afcd9dbeaf01/users/b5c29d7121a44c3f85abc61d1eda8320/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:40.300876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c505c8-ef4c-40ac-9cc0-b04cc700fa5d None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:40.301174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c505c8-ef4c-40ac-9cc0-b04cc700fa5d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:40.301550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf2231-7f77-41cc-9376-744a51ae664e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de8e4142bed424d8845afcd9dbeaf01, user_id=b5c29d7121a44c3f85abc61d1eda8320, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:40.301683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c505c8-ef4c-40ac-9cc0-b04cc700fa5d None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:40.302982 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 817/1640] 10.222.0.33 () {54 vars in 912 bytes} [Fri Jun 17 07:19:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:19:40.309363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69161457-99a6-4e02-9eb8-8ca0907f55f0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:40.309573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69161457-99a6-4e02-9eb8-8ca0907f55f0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:40.309857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69161457-99a6-4e02-9eb8-8ca0907f55f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:40.310183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf2231-7f77-41cc-9376-744a51ae664e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:40.322527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 824/1641] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:40 2022] PUT /identity/v3/projects/0de8e4142bed424d8845afcd9dbeaf01/users/b5c29d7121a44c3f85abc61d1eda8320/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:40.336507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edd279-bb17-4361-8f6f-82d53135053d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:40.336799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edd279-bb17-4361-8f6f-82d53135053d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:40.337078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edd279-bb17-4361-8f6f-82d53135053d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:40.362487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-69161457-99a6-4e02-9eb8-8ca0907f55f0 None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:40.363735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-69161457-99a6-4e02-9eb8-8ca0907f55f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:40.417375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-96edd279-bb17-4361-8f6f-82d53135053d None None] MFA Rules not processed for user `b5c29d7121a44c3f85abc61d1eda8320`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:40.419784 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-96edd279-bb17-4361-8f6f-82d53135053d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:40.435810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:40.533523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 818/1642] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:19:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:40.535366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69161457-99a6-4e02-9eb8-8ca0907f55f0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:40.539364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-69161457-99a6-4e02-9eb8-8ca0907f55f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:40.596387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 825/1643] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:40.601448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96edd279-bb17-4361-8f6f-82d53135053d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:40.610438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-96edd279-bb17-4361-8f6f-82d53135053d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:40.685252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69161457-99a6-4e02-9eb8-8ca0907f55f0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:40.696428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a73694-1e68-45cb-b1f1-ecd3928aeefc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:40.696774 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a73694-1e68-45cb-b1f1-ecd3928aeefc service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:40.697826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a73694-1e68-45cb-b1f1-ecd3928aeefc service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:40.698979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a73694-1e68-45cb-b1f1-ecd3928aeefc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:40.769349 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96edd279-bb17-4361-8f6f-82d53135053d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:40.784006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b643a-1a49-443e-b8c3-ce7ae5b7d234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:40.784721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b643a-1a49-443e-b8c3-ce7ae5b7d234 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:40.785155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b643a-1a49-443e-b8c3-ce7ae5b7d234 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:40.785447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b643a-1a49-443e-b8c3-ce7ae5b7d234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:40.808380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b643a-1a49-443e-b8c3-ce7ae5b7d234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:40.813481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b643a-1a49-443e-b8c3-ce7ae5b7d234 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:40.852865 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 826/1644] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:40.865460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a73694-1e68-45cb-b1f1-ecd3928aeefc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:40.868433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a73694-1e68-45cb-b1f1-ecd3928aeefc service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:40.887128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:40.914257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 819/1645] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:40.930970 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b643a-1a49-443e-b8c3-ce7ae5b7d234 service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:40.933152 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-cd9b643a-1a49-443e-b8c3-ce7ae5b7d234 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:40.945660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b643a-1a49-443e-b8c3-ce7ae5b7d234 service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:40.952798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb066cf-9758-44ae-b477-253d4d882d97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:40.953624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb066cf-9758-44ae-b477-253d4d882d97 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:40.953951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb066cf-9758-44ae-b477-253d4d882d97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:40.954232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb066cf-9758-44ae-b477-253d4d882d97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:40.956517 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-1fb066cf-9758-44ae-b477-253d4d882d97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:41.113435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb066cf-9758-44ae-b477-253d4d882d97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:41.114539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb066cf-9758-44ae-b477-253d4d882d97 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:41.144939 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 827/1646] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:41.292895 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a73694-1e68-45cb-b1f1-ecd3928aeefc service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:41.294394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-32a73694-1e68-45cb-b1f1-ecd3928aeefc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:41.337900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a73694-1e68-45cb-b1f1-ecd3928aeefc service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:41.377986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:41.397177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:41.397516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:41.397802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] PATH_INFO: `/v3/users/ad2d4ce6c4b840b78abc19c28063a747` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:41.404388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad2d4ce6c4b840b78abc19c28063a747)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:41.410124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:41.542748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad2d4ce6c4b840b78abc19c28063a747'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:41.560135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad2d4ce6c4b840b78abc19c28063a747'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:41.561001 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad2d4ce6c4b840b78abc19c28063a747'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:41.594079 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad2d4ce6c4b840b78abc19c28063a747'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:41.594848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad2d4ce6c4b840b78abc19c28063a747'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:41.619836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 820/1647] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:19:41 2022] DELETE /identity/v3/users/ad2d4ce6c4b840b78abc19c28063a747 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:41.624483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c3bb5-835b-4229-b835-99ff89c5ff6b None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:41.625049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c3bb5-835b-4229-b835-99ff89c5ff6b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:41.625049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c3bb5-835b-4229-b835-99ff89c5ff6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:41.656257 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-d31c3bb5-835b-4229-b835-99ff89c5ff6b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:41.659267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d31c3bb5-835b-4229-b835-99ff89c5ff6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:41.753728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:41.807439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 828/1648] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:41.856554 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:41.858153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:42.015204 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306a969-2aaa-43c0-9021-7703c4abb416 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:42.032910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a002489-c24e-4e9e-b6be-c106b875dd76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:42.033885 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a002489-c24e-4e9e-b6be-c106b875dd76 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:42.034286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a002489-c24e-4e9e-b6be-c106b875dd76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:42.035367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a002489-c24e-4e9e-b6be-c106b875dd76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:42.063384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a002489-c24e-4e9e-b6be-c106b875dd76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:42.069174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a002489-c24e-4e9e-b6be-c106b875dd76 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:42.120097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 821/1649] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:43.090926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c3bb5-835b-4229-b835-99ff89c5ff6b None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:43.092490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d31c3bb5-835b-4229-b835-99ff89c5ff6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:43.108214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c3bb5-835b-4229-b835-99ff89c5ff6b None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:43.116276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:43.117375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:43.117881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:43.118300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf admin admin] PATH_INFO: `/v3/projects/7bdb0b874c434af39fb644908137de77` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:43.120330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bdb0b874c434af39fb644908137de77)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:43.123710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:43.284946 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bdb0b874c434af39fb644908137de77', 'request_id': 'req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:43.308886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bdb0b874c434af39fb644908137de77', 'request_id': 'req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:43.323419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bdb0b874c434af39fb644908137de77', 'request_id': 'req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:43.353967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf admin admin] The token cache is being invalidate because project 7bdb0b874c434af39fb644908137de77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:19:43.354634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:43.355503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-3cd14fc3-f10a-4b34-a14f-dc8ebfad5caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:43.360830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 829/1650] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:19:43 2022] DELETE /identity/v3/projects/7bdb0b874c434af39fb644908137de77 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:43.362548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d840e-35d7-4a02-b812-d89ae3373b95 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:43.365102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d840e-35d7-4a02-b812-d89ae3373b95 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:43.365400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d840e-35d7-4a02-b812-d89ae3373b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:43.408476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-669d840e-35d7-4a02-b812-d89ae3373b95 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:43.413512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-669d840e-35d7-4a02-b812-d89ae3373b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:43.588629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 822/1651] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:43.598374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716dcc0-68c8-4d43-ba6a-8674510f1795 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:43.599221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716dcc0-68c8-4d43-ba6a-8674510f1795 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:43.599612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716dcc0-68c8-4d43-ba6a-8674510f1795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:43.629123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-6716dcc0-68c8-4d43-ba6a-8674510f1795 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:43.634744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6716dcc0-68c8-4d43-ba6a-8674510f1795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:43.661250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 830/1652] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:43.661695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d840e-35d7-4a02-b812-d89ae3373b95 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:43.722126 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d840e-35d7-4a02-b812-d89ae3373b95 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:43.739712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4f54d-cbdc-4959-a8ab-00bc1ca53410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:43.741204 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4f54d-cbdc-4959-a8ab-00bc1ca53410 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:43.741605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4f54d-cbdc-4959-a8ab-00bc1ca53410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:43.744179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4f54d-cbdc-4959-a8ab-00bc1ca53410 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:43.745074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4f54d-cbdc-4959-a8ab-00bc1ca53410 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:19:43.745509 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4f54d-cbdc-4959-a8ab-00bc1ca53410 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:43.747257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4f54d-cbdc-4959-a8ab-00bc1ca53410 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:43.779468 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:43.804716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716dcc0-68c8-4d43-ba6a-8674510f1795 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:43.820195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 823/1653] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:19:43.821549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716dcc0-68c8-4d43-ba6a-8674510f1795 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:43.836009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91761d25-7a0f-4b86-8ac3-57e2dafdd788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:43.836877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91761d25-7a0f-4b86-8ac3-57e2dafdd788 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:43.839454 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91761d25-7a0f-4b86-8ac3-57e2dafdd788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:43.839858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91761d25-7a0f-4b86-8ac3-57e2dafdd788 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:43.841910 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91761d25-7a0f-4b86-8ac3-57e2dafdd788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:43.849634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35836-3ccf-40bc-8726-64821eb109c6 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:43.850007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35836-3ccf-40bc-8726-64821eb109c6 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:43.850361 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35836-3ccf-40bc-8726-64821eb109c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:43.858638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91761d25-7a0f-4b86-8ac3-57e2dafdd788 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:43.903823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-2bf35836-3ccf-40bc-8726-64821eb109c6 None None] MFA Rules not processed for user `b3ca63f8f8484ddf9d7c72749ba7f482`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:43.909742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-2bf35836-3ccf-40bc-8726-64821eb109c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:43.947463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 831/1654] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:43 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:43.952765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91761d25-7a0f-4b86-8ac3-57e2dafdd788 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:43.959664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-91761d25-7a0f-4b86-8ac3-57e2dafdd788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:44.147184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 824/1655] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:44.147184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf35836-3ccf-40bc-8726-64821eb109c6 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:44.163381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf35836-3ccf-40bc-8726-64821eb109c6 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:44.175417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f2ec6-3aa8-42ce-ba17-48e1930cbf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:44.176170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f2ec6-3aa8-42ce-ba17-48e1930cbf26 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:44.176467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f2ec6-3aa8-42ce-ba17-48e1930cbf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:44.176665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91761d25-7a0f-4b86-8ac3-57e2dafdd788 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:44.177102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f2ec6-3aa8-42ce-ba17-48e1930cbf26 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:44.177440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f2ec6-3aa8-42ce-ba17-48e1930cbf26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:44.185316 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f2ec6-3aa8-42ce-ba17-48e1930cbf26 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:44.186908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:44.187936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:44.188850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:44.189311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] PATH_INFO: `/v3/users/3d519a07e8434aaa95adb641b669838f` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:44.191625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d519a07e8434aaa95adb641b669838f)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:44.198783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:44.228852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 825/1656] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:44 2022] POST /identity/v3/users => generated 291 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:44.229718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f2ec6-3aa8-42ce-ba17-48e1930cbf26 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:44.240990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3e1f2ec6-3aa8-42ce-ba17-48e1930cbf26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:44.282948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f2ec6-3aa8-42ce-ba17-48e1930cbf26 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:44.285513 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:44.297735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:44.298639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:44.298923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:44.299253 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] PATH_INFO: `/v3/projects/73cc2dce7dd1435ca89a36e3befc27aa` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:44.300660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73cc2dce7dd1435ca89a36e3befc27aa)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:44.301713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: WARNING py.warnings [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 07:19:44.301713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: warnings.warn(msg) Jun 17 07:19:44.301713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: Jun 17 07:19:44.302946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:44.305219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d519a07e8434aaa95adb641b669838f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.308028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d519a07e8434aaa95adb641b669838f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.328728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d519a07e8434aaa95adb641b669838f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.350720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d519a07e8434aaa95adb641b669838f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.351728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d519a07e8434aaa95adb641b669838f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.375749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 832/1657] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:19:43 2022] DELETE /identity/v3/users/3d519a07e8434aaa95adb641b669838f => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:44.376964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:44.379699 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:44.415874 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73cc2dce7dd1435ca89a36e3befc27aa', 'request_id': 'req-165d4b8d-3358-413d-b5d2-6a4e63b257c0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.439526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73cc2dce7dd1435ca89a36e3befc27aa', 'request_id': 'req-165d4b8d-3358-413d-b5d2-6a4e63b257c0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.481511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73cc2dce7dd1435ca89a36e3befc27aa', 'request_id': 'req-165d4b8d-3358-413d-b5d2-6a4e63b257c0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.511600 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027db748-65db-4dd7-8af3-393b9bcafe03 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:44.518033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477ea65-87ee-4679-ae01-4f48deb8ea37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:44.518417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477ea65-87ee-4679-ae01-4f48deb8ea37 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:44.518810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477ea65-87ee-4679-ae01-4f48deb8ea37 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:44.519189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477ea65-87ee-4679-ae01-4f48deb8ea37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:44.521320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-2477ea65-87ee-4679-ae01-4f48deb8ea37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:44.523870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] The token cache is being invalidate because project 73cc2dce7dd1435ca89a36e3befc27aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:19:44.524550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.527178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.541480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 826/1658] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:19:44 2022] DELETE /identity/v3/projects/73cc2dce7dd1435ca89a36e3befc27aa => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:44.542704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:44.544371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:44.607967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477ea65-87ee-4679-ae01-4f48deb8ea37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:44.612529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477ea65-87ee-4679-ae01-4f48deb8ea37 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:44.640000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:44.642117 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 833/1659] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:44.647435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce200fb-a35d-475d-9d9a-f94aaf239ca9 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:44.648025 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d4b8d-3358-413d-b5d2-6a4e63b257c0 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:44.648374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce200fb-a35d-475d-9d9a-f94aaf239ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:44.648797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce200fb-a35d-475d-9d9a-f94aaf239ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:44.695880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-2ce200fb-a35d-475d-9d9a-f94aaf239ca9 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:44.698126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-2ce200fb-a35d-475d-9d9a-f94aaf239ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:44.745282 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8977ec-0e77-4d79-b69d-43c59892154d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:44.746335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8977ec-0e77-4d79-b69d-43c59892154d admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:44.746859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8977ec-0e77-4d79-b69d-43c59892154d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:44.747632 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8977ec-0e77-4d79-b69d-43c59892154d admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:44.748328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8977ec-0e77-4d79-b69d-43c59892154d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:44.752163 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8977ec-0e77-4d79-b69d-43c59892154d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:44.772255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 827/1660] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:44.773281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8977ec-0e77-4d79-b69d-43c59892154d admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:44.775261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-0a8977ec-0e77-4d79-b69d-43c59892154d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:44.796721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8977ec-0e77-4d79-b69d-43c59892154d admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:44.803786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7657b7-8351-4bc6-bbe4-fb44ecd730fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:44.805069 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7657b7-8351-4bc6-bbe4-fb44ecd730fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:44.805458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7657b7-8351-4bc6-bbe4-fb44ecd730fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:44.805848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7657b7-8351-4bc6-bbe4-fb44ecd730fe admin admin] PATH_INFO: `/v3/users/bbce34136ede42029c45a6b3c89b77b0` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:44.808684 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7657b7-8351-4bc6-bbe4-fb44ecd730fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbce34136ede42029c45a6b3c89b77b0)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:44.811091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7657b7-8351-4bc6-bbe4-fb44ecd730fe admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:44.841132 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:44.882999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 834/1661] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:44.887501 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce200fb-a35d-475d-9d9a-f94aaf239ca9 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:44.909103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce200fb-a35d-475d-9d9a-f94aaf239ca9 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:44.921920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83bd5b-5b50-4e58-8fae-8702309960ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:44.922617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83bd5b-5b50-4e58-8fae-8702309960ec admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:44.922902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83bd5b-5b50-4e58-8fae-8702309960ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:44.923179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83bd5b-5b50-4e58-8fae-8702309960ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:44.923919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83bd5b-5b50-4e58-8fae-8702309960ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:44.925585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83bd5b-5b50-4e58-8fae-8702309960ec admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:44.944417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8f7657b7-8351-4bc6-bbe4-fb44ecd730fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbce34136ede42029c45a6b3c89b77b0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.945811 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 835/1662] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:44.949976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83bd5b-5b50-4e58-8fae-8702309960ec admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:44.957920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8f7657b7-8351-4bc6-bbe4-fb44ecd730fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbce34136ede42029c45a6b3c89b77b0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.958844 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8f7657b7-8351-4bc6-bbe4-fb44ecd730fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbce34136ede42029c45a6b3c89b77b0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.967876 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83bd5b-5b50-4e58-8fae-8702309960ec admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:44.975174 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccf8de-0623-49ad-b671-b366e6c562e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:44.976881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccf8de-0623-49ad-b671-b366e6c562e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:44.977179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccf8de-0623-49ad-b671-b366e6c562e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:44.977433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccf8de-0623-49ad-b671-b366e6c562e0 admin admin] PATH_INFO: `/v3/projects/05e314048d714af09aca36b6c1b15cc2/users/67b63ee1c8144b66b001ba5ce1398fc5/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:44.989237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8f7657b7-8351-4bc6-bbe4-fb44ecd730fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbce34136ede42029c45a6b3c89b77b0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:44.990019 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8f7657b7-8351-4bc6-bbe4-fb44ecd730fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbce34136ede42029c45a6b3c89b77b0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:45.009105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccf8de-0623-49ad-b671-b366e6c562e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e314048d714af09aca36b6c1b15cc2, user_id=67b63ee1c8144b66b001ba5ce1398fc5, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:45.012399 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccf8de-0623-49ad-b671-b366e6c562e0 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:45.048929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 836/1663] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:44 2022] PUT /identity/v3/projects/05e314048d714af09aca36b6c1b15cc2/users/67b63ee1c8144b66b001ba5ce1398fc5/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:45.048929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccf8de-0623-49ad-b671-b366e6c562e0 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:45.049086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 828/1664] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:19:44 2022] DELETE /identity/v3/users/bbce34136ede42029c45a6b3c89b77b0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:45.051691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9eaba-32df-467a-b0a7-a22e881db80a None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:45.051829 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9eaba-32df-467a-b0a7-a22e881db80a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:45.052098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9eaba-32df-467a-b0a7-a22e881db80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:45.054648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-feccf8de-0623-49ad-b671-b366e6c562e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:45.111609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-58a9eaba-32df-467a-b0a7-a22e881db80a None None] MFA Rules not processed for user `67b63ee1c8144b66b001ba5ce1398fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:45.112923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-58a9eaba-32df-467a-b0a7-a22e881db80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:45.179737 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccf8de-0623-49ad-b671-b366e6c562e0 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:45.243936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:45.252812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 829/1665] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:45.253793 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9eaba-32df-467a-b0a7-a22e881db80a None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:45.255787 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-58a9eaba-32df-467a-b0a7-a22e881db80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:45.261224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80ae2b-ca00-4b1c-92a1-71d83a66ce9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:45.263034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80ae2b-ca00-4b1c-92a1-71d83a66ce9d service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:45.263331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80ae2b-ca00-4b1c-92a1-71d83a66ce9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:45.263613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80ae2b-ca00-4b1c-92a1-71d83a66ce9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:45.266031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-1c80ae2b-ca00-4b1c-92a1-71d83a66ce9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:45.346223 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9eaba-32df-467a-b0a7-a22e881db80a None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:45.349894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80ae2b-ca00-4b1c-92a1-71d83a66ce9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:45.351661 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80ae2b-ca00-4b1c-92a1-71d83a66ce9d service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:45.358811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362732c-c6ee-4bcd-a32b-99fd354e1697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:45.359833 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362732c-c6ee-4bcd-a32b-99fd354e1697 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:45.361035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362732c-c6ee-4bcd-a32b-99fd354e1697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:45.361378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362732c-c6ee-4bcd-a32b-99fd354e1697 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:45.362094 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362732c-c6ee-4bcd-a32b-99fd354e1697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:45.365913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 837/1666] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:45.367348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362732c-c6ee-4bcd-a32b-99fd354e1697 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:45.421489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80ae2b-ca00-4b1c-92a1-71d83a66ce9d service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:45.432388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 830/1667] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:45 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:45.434424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80ae2b-ca00-4b1c-92a1-71d83a66ce9d service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:45.470370 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:45.517532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a332f-34fd-4de6-86aa-7eadaaf806d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:45.520820 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a332f-34fd-4de6-86aa-7eadaaf806d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:45.521741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a332f-34fd-4de6-86aa-7eadaaf806d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:45.521741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a332f-34fd-4de6-86aa-7eadaaf806d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:45.527812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a332f-34fd-4de6-86aa-7eadaaf806d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:45.530898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a332f-34fd-4de6-86aa-7eadaaf806d7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:45.586891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362732c-c6ee-4bcd-a32b-99fd354e1697 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:45.588435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 838/1668] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:45 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:45.637490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362732c-c6ee-4bcd-a32b-99fd354e1697 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:45.657620 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b67bb-1232-464f-9761-e42e6e398721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:45.658791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b67bb-1232-464f-9761-e42e6e398721 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:45.659345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b67bb-1232-464f-9761-e42e6e398721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:45.659843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b67bb-1232-464f-9761-e42e6e398721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:45.662147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b67bb-1232-464f-9761-e42e6e398721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:45.673818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b67bb-1232-464f-9761-e42e6e398721 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:45.712513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 831/1669] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:45.712891 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a332f-34fd-4de6-86aa-7eadaaf806d7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:45.731847 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a332f-34fd-4de6-86aa-7eadaaf806d7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:45.745252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a1aa6-48c6-4d6b-b5cd-59c2573cd610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:45.747103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a1aa6-48c6-4d6b-b5cd-59c2573cd610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:45.747510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a1aa6-48c6-4d6b-b5cd-59c2573cd610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:45.747903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a1aa6-48c6-4d6b-b5cd-59c2573cd610 admin admin] PATH_INFO: `/v3/projects/452c022a71a64a50b9123bfee135cb56/users/af4d8a80ff3e43ba9aba1b7d926d3ac2/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:45.786835 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a1aa6-48c6-4d6b-b5cd-59c2573cd610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=452c022a71a64a50b9123bfee135cb56, user_id=af4d8a80ff3e43ba9aba1b7d926d3ac2, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:45.794445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a1aa6-48c6-4d6b-b5cd-59c2573cd610 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:45.825210 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 839/1670] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:45 2022] PUT /identity/v3/projects/452c022a71a64a50b9123bfee135cb56/users/af4d8a80ff3e43ba9aba1b7d926d3ac2/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:19:45.832367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b67bb-1232-464f-9761-e42e6e398721 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:45.875044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b67bb-1232-464f-9761-e42e6e398721 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:45.980973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d651c-68c1-4570-9b46-9ab37d5ae755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:45.985559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d651c-68c1-4570-9b46-9ab37d5ae755 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:45.986510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d651c-68c1-4570-9b46-9ab37d5ae755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:45.987092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d651c-68c1-4570-9b46-9ab37d5ae755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:45.987893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d651c-68c1-4570-9b46-9ab37d5ae755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:45.991098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d651c-68c1-4570-9b46-9ab37d5ae755 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:46.012094 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 832/1671] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:46.012829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a1aa6-48c6-4d6b-b5cd-59c2573cd610 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:46.028499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a1aa6-48c6-4d6b-b5cd-59c2573cd610 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:46.036711 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777714a0-5dc0-41c9-884d-eab134618bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:46.037859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777714a0-5dc0-41c9-884d-eab134618bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:46.038356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777714a0-5dc0-41c9-884d-eab134618bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:46.038756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777714a0-5dc0-41c9-884d-eab134618bd6 admin admin] PATH_INFO: `/v3/projects/452c022a71a64a50b9123bfee135cb56/users/af4d8a80ff3e43ba9aba1b7d926d3ac2/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:46.041871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777714a0-5dc0-41c9-884d-eab134618bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=452c022a71a64a50b9123bfee135cb56, user_id=af4d8a80ff3e43ba9aba1b7d926d3ac2, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:46.047169 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777714a0-5dc0-41c9-884d-eab134618bd6 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:46.069961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 840/1672] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:46 2022] PUT /identity/v3/projects/452c022a71a64a50b9123bfee135cb56/users/af4d8a80ff3e43ba9aba1b7d926d3ac2/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:19:46.081480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453304c4-96b7-4824-a227-54d91f7cd0d0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:46.081786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453304c4-96b7-4824-a227-54d91f7cd0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:46.082075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453304c4-96b7-4824-a227-54d91f7cd0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:46.118445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-453304c4-96b7-4824-a227-54d91f7cd0d0 None None] MFA Rules not processed for user `af4d8a80ff3e43ba9aba1b7d926d3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:46.121970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-453304c4-96b7-4824-a227-54d91f7cd0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:46.190601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777714a0-5dc0-41c9-884d-eab134618bd6 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:46.194056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-777714a0-5dc0-41c9-884d-eab134618bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:46.251834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:46.273865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 833/1673] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:46.281682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453304c4-96b7-4824-a227-54d91f7cd0d0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:46.284195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-453304c4-96b7-4824-a227-54d91f7cd0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:46.382493 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777714a0-5dc0-41c9-884d-eab134618bd6 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:46.396824 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:46.398144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:46.398582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:46.406438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] PATH_INFO: `/v3/projects/ca5bba369109460b8a44d927072bd277` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:46.407565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5bba369109460b8a44d927072bd277)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:46.409709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:46.426234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453304c4-96b7-4824-a227-54d91f7cd0d0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:46.432182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:46.433106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:46.433627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:46.433794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c admin admin] PATH_INFO: `/v3/projects/96d2a4641fe84ed396e4cd1acb17ab34` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:46.435744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=96d2a4641fe84ed396e4cd1acb17ab34)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:46.438754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:46.514906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5bba369109460b8a44d927072bd277', 'request_id': 'req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:46.522788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96d2a4641fe84ed396e4cd1acb17ab34', 'request_id': 'req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:46.533355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5bba369109460b8a44d927072bd277', 'request_id': 'req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:46.534959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96d2a4641fe84ed396e4cd1acb17ab34', 'request_id': 'req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:46.545526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96d2a4641fe84ed396e4cd1acb17ab34', 'request_id': 'req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:46.546285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5bba369109460b8a44d927072bd277', 'request_id': 'req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:46.569465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] The token cache is being invalidate because project ca5bba369109460b8a44d927072bd277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:19:46.569991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:46.571048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:46.574862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 841/1674] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:19:46 2022] DELETE /identity/v3/projects/ca5bba369109460b8a44d927072bd277 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:46.575902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:46.581275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:46.584528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c admin admin] The token cache is being invalidate because project 96d2a4641fe84ed396e4cd1acb17ab34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:19:46.584835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:46.585561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1d75fc8b-c1c1-4cd3-9906-5ed810249a4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:46.588295 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 834/1675] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:19:46 2022] DELETE /identity/v3/projects/96d2a4641fe84ed396e4cd1acb17ab34 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:46.592617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29593521-ab7a-4840-b270-3c35cd44a06b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:46.593275 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29593521-ab7a-4840-b270-3c35cd44a06b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:46.594624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29593521-ab7a-4840-b270-3c35cd44a06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:46.608427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:46.647468 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-29593521-ab7a-4840-b270-3c35cd44a06b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:46.649044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-29593521-ab7a-4840-b270-3c35cd44a06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:46.745364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd6afb-b140-4f7f-bbac-3f2aedacbafb admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:46.785124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 835/1676] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:46.788272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1648b33-7f08-4c0d-abb5-ee853c7bced6 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:46.788272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1648b33-7f08-4c0d-abb5-ee853c7bced6 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:46.788272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1648b33-7f08-4c0d-abb5-ee853c7bced6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:46.819651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c1648b33-7f08-4c0d-abb5-ee853c7bced6 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:46.821086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c1648b33-7f08-4c0d-abb5-ee853c7bced6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:46.837234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 836/1677] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:46.845084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067f761-38b0-4bbd-ac5d-302854677eb2 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:46.845391 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067f761-38b0-4bbd-ac5d-302854677eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:46.845677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067f761-38b0-4bbd-ac5d-302854677eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:46.892280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d709c87-4319-4d0d-8ef2-f3741a912e9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:46.893354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d709c87-4319-4d0d-8ef2-f3741a912e9b service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:46.893463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6067f761-38b0-4bbd-ac5d-302854677eb2 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:46.897158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6067f761-38b0-4bbd-ac5d-302854677eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:46.906496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d709c87-4319-4d0d-8ef2-f3741a912e9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:46.907106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d709c87-4319-4d0d-8ef2-f3741a912e9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:46.911338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-4d709c87-4319-4d0d-8ef2-f3741a912e9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:46.922084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 837/1678] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:46.942927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db550b5e-afff-49c5-be51-90d157e66605 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:46.943245 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db550b5e-afff-49c5-be51-90d157e66605 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:46.943534 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db550b5e-afff-49c5-be51-90d157e66605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:46.973988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-db550b5e-afff-49c5-be51-90d157e66605 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:46.975244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-db550b5e-afff-49c5-be51-90d157e66605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:46.988058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 838/1679] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:46.988984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db550b5e-afff-49c5-be51-90d157e66605 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.004679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db550b5e-afff-49c5-be51-90d157e66605 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.013268 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f40289-263c-4ad4-882a-062028a5daf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.014106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f40289-263c-4ad4-882a-062028a5daf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.014471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f40289-263c-4ad4-882a-062028a5daf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.014827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f40289-263c-4ad4-882a-062028a5daf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.015675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f40289-263c-4ad4-882a-062028a5daf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:19:47.016090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f40289-263c-4ad4-882a-062028a5daf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.018102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f40289-263c-4ad4-882a-062028a5daf9 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.019975 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:47.032449 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d709c87-4319-4d0d-8ef2-f3741a912e9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.035035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d709c87-4319-4d0d-8ef2-f3741a912e9b service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.042607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 839/1680] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:19:47.042607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f40289-263c-4ad4-882a-062028a5daf9 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.056209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f40289-263c-4ad4-882a-062028a5daf9 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.057526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 842/1681] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:47.058592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d709c87-4319-4d0d-8ef2-f3741a912e9b service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.062340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421d6c7-84d5-4c2a-989d-19068f673b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.063023 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421d6c7-84d5-4c2a-989d-19068f673b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.065991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421d6c7-84d5-4c2a-989d-19068f673b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.066293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421d6c7-84d5-4c2a-989d-19068f673b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.069574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421d6c7-84d5-4c2a-989d-19068f673b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:19:47.069909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421d6c7-84d5-4c2a-989d-19068f673b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.074293 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d709c87-4319-4d0d-8ef2-f3741a912e9b service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.075663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421d6c7-84d5-4c2a-989d-19068f673b3a admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.082102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fe159-dbe2-4d74-8916-18fc005b0650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.083060 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fe159-dbe2-4d74-8916-18fc005b0650 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.083634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fe159-dbe2-4d74-8916-18fc005b0650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.084161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fe159-dbe2-4d74-8916-18fc005b0650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.085312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fe159-dbe2-4d74-8916-18fc005b0650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.088800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fe159-dbe2-4d74-8916-18fc005b0650 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.108828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 840/1682] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:19:47.113015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421d6c7-84d5-4c2a-989d-19068f673b3a admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.127041 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421d6c7-84d5-4c2a-989d-19068f673b3a admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.132913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28729b4c-cb2e-4e07-a4da-1d44d3e5e411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.133603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729b4c-cb2e-4e07-a4da-1d44d3e5e411 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.133899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729b4c-cb2e-4e07-a4da-1d44d3e5e411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.134184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729b4c-cb2e-4e07-a4da-1d44d3e5e411 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.135427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28729b4c-cb2e-4e07-a4da-1d44d3e5e411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.137432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28729b4c-cb2e-4e07-a4da-1d44d3e5e411 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.152696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 843/1683] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:47 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:47.153664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fe159-dbe2-4d74-8916-18fc005b0650 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.201582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 841/1684] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:47 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:19:47.202855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28729b4c-cb2e-4e07-a4da-1d44d3e5e411 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.222263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:47.235480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fe159-dbe2-4d74-8916-18fc005b0650 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.333292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28729b4c-cb2e-4e07-a4da-1d44d3e5e411 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.333763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb1796-b968-4b8e-b7b0-2797854a3b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.334510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb1796-b968-4b8e-b7b0-2797854a3b8f admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.334989 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb1796-b968-4b8e-b7b0-2797854a3b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.335512 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb1796-b968-4b8e-b7b0-2797854a3b8f admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.336306 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb1796-b968-4b8e-b7b0-2797854a3b8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.339884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb1796-b968-4b8e-b7b0-2797854a3b8f admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.344085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864473a4-8011-4ab1-8aa4-70d13945400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.345595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864473a4-8011-4ab1-8aa4-70d13945400f admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.345893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864473a4-8011-4ab1-8aa4-70d13945400f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.346182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864473a4-8011-4ab1-8aa4-70d13945400f admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.346897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864473a4-8011-4ab1-8aa4-70d13945400f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.350541 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864473a4-8011-4ab1-8aa4-70d13945400f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.400849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 842/1685] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:47 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:47.401946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864473a4-8011-4ab1-8aa4-70d13945400f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.405566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 844/1686] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:47 2022] POST /identity/v3/users => generated 294 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:47.408663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb1796-b968-4b8e-b7b0-2797854a3b8f admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.409606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-864473a4-8011-4ab1-8aa4-70d13945400f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:47.435486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb1796-b968-4b8e-b7b0-2797854a3b8f admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.459786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba17214-61cf-431f-9b77-e9ad6f15bfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.460590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba17214-61cf-431f-9b77-e9ad6f15bfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.460957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba17214-61cf-431f-9b77-e9ad6f15bfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.461364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba17214-61cf-431f-9b77-e9ad6f15bfa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.462189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba17214-61cf-431f-9b77-e9ad6f15bfa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.463917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba17214-61cf-431f-9b77-e9ad6f15bfa6 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.491024 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 845/1687] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:47.491580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba17214-61cf-431f-9b77-e9ad6f15bfa6 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.522771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba17214-61cf-431f-9b77-e9ad6f15bfa6 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.534734 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d195251-5ee4-4318-8e03-2b85b6e6532d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.535599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d195251-5ee4-4318-8e03-2b85b6e6532d admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.536262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d195251-5ee4-4318-8e03-2b85b6e6532d admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.536661 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d195251-5ee4-4318-8e03-2b85b6e6532d admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.538526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d195251-5ee4-4318-8e03-2b85b6e6532d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.545163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d195251-5ee4-4318-8e03-2b85b6e6532d admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.560728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864473a4-8011-4ab1-8aa4-70d13945400f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.567517 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 846/1688] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:47.568815 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d195251-5ee4-4318-8e03-2b85b6e6532d admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.572431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-2d195251-5ee4-4318-8e03-2b85b6e6532d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:47.572726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17826d-bacd-4228-bf44-2a0303997d37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.573421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17826d-bacd-4228-bf44-2a0303997d37 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.573711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17826d-bacd-4228-bf44-2a0303997d37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.574051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17826d-bacd-4228-bf44-2a0303997d37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.580347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-aa17826d-bacd-4228-bf44-2a0303997d37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:47.590295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d195251-5ee4-4318-8e03-2b85b6e6532d admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.595148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b4480-427c-45ab-bc3e-350a28ea0ac8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.596236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b4480-427c-45ab-bc3e-350a28ea0ac8 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.597232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b4480-427c-45ab-bc3e-350a28ea0ac8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.597887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b4480-427c-45ab-bc3e-350a28ea0ac8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.600269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-179b4480-427c-45ab-bc3e-350a28ea0ac8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:47.710882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17826d-bacd-4228-bf44-2a0303997d37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.712453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17826d-bacd-4228-bf44-2a0303997d37 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.725221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b4480-427c-45ab-bc3e-350a28ea0ac8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.727295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b4480-427c-45ab-bc3e-350a28ea0ac8 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.737289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 843/1689] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:47.740441 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17826d-bacd-4228-bf44-2a0303997d37 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.752417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 847/1690] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:47.752989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:47.753268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b4480-427c-45ab-bc3e-350a28ea0ac8 service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.772944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17826d-bacd-4228-bf44-2a0303997d37 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.782164 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b3fed-4505-4482-960b-7d0f857bc88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.783224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b3fed-4505-4482-960b-7d0f857bc88f admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.783722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b3fed-4505-4482-960b-7d0f857bc88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.784261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b3fed-4505-4482-960b-7d0f857bc88f admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.785218 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b3fed-4505-4482-960b-7d0f857bc88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.787540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b3fed-4505-4482-960b-7d0f857bc88f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.793878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b4480-427c-45ab-bc3e-350a28ea0ac8 service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.807954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 844/1691] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:47.809141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae610c8-eb0f-4296-881a-73badcb25d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.811074 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae610c8-eb0f-4296-881a-73badcb25d7c admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.811585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae610c8-eb0f-4296-881a-73badcb25d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.812099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae610c8-eb0f-4296-881a-73badcb25d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.813189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae610c8-eb0f-4296-881a-73badcb25d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.814348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b3fed-4505-4482-960b-7d0f857bc88f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.817394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae610c8-eb0f-4296-881a-73badcb25d7c admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.844593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b3fed-4505-4482-960b-7d0f857bc88f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.854579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 848/1692] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:47.859702 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae610c8-eb0f-4296-881a-73badcb25d7c admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:47.864865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4d349-2557-4a2d-be1d-84947d59b59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.871982 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4d349-2557-4a2d-be1d-84947d59b59a admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.872500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4d349-2557-4a2d-be1d-84947d59b59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.872900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4d349-2557-4a2d-be1d-84947d59b59a admin admin] PATH_INFO: `/v3/projects/362b426c8f7d4bf7a292aed5d1c0eaf6/users/b41a1927cdd64ee88cb7974ef48d07bd/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.886547 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae610c8-eb0f-4296-881a-73badcb25d7c admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:47.896174 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43281d2c-5f7e-42c6-ae62-35dcf692299d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:47.901458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43281d2c-5f7e-42c6-ae62-35dcf692299d admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.901859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43281d2c-5f7e-42c6-ae62-35dcf692299d admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.902160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43281d2c-5f7e-42c6-ae62-35dcf692299d admin admin] PATH_INFO: `/v3/projects/41cc777c1a32431087757206d5f27e6b/users/a343b5e63854409681f9b4d8c7f70836/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.908413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4d349-2557-4a2d-be1d-84947d59b59a admin admin] RBAC: Authorizing `identity:create_grant(project_id=362b426c8f7d4bf7a292aed5d1c0eaf6, user_id=b41a1927cdd64ee88cb7974ef48d07bd, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.911831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4d349-2557-4a2d-be1d-84947d59b59a admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.927412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43281d2c-5f7e-42c6-ae62-35dcf692299d admin admin] RBAC: Authorizing `identity:create_grant(project_id=41cc777c1a32431087757206d5f27e6b, user_id=a343b5e63854409681f9b4d8c7f70836, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:47.932107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43281d2c-5f7e-42c6-ae62-35dcf692299d admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:47.946605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 845/1693] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:47 2022] PUT /identity/v3/projects/362b426c8f7d4bf7a292aed5d1c0eaf6/users/b41a1927cdd64ee88cb7974ef48d07bd/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:47.956572 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae363cb-6ba8-43fa-a9b9-3fe32fe0a4b6 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.957114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 849/1694] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:47 2022] PUT /identity/v3/projects/41cc777c1a32431087757206d5f27e6b/users/a343b5e63854409681f9b4d8c7f70836/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:47.957602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae363cb-6ba8-43fa-a9b9-3fe32fe0a4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.958098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae363cb-6ba8-43fa-a9b9-3fe32fe0a4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:47.970433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69b2a-7ff6-41d5-85ba-b6a64ffb8eca None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:47.971134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69b2a-7ff6-41d5-85ba-b6a64ffb8eca None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:47.971660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69b2a-7ff6-41d5-85ba-b6a64ffb8eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:48.010824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-cae363cb-6ba8-43fa-a9b9-3fe32fe0a4b6 None None] MFA Rules not processed for user `b41a1927cdd64ee88cb7974ef48d07bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:48.013157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cae363cb-6ba8-43fa-a9b9-3fe32fe0a4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:48.038441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-40a69b2a-7ff6-41d5-85ba-b6a64ffb8eca None None] MFA Rules not processed for user `a343b5e63854409681f9b4d8c7f70836`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:48.039693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-40a69b2a-7ff6-41d5-85ba-b6a64ffb8eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:48.079269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:48.110739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 846/1695] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:48.111958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae363cb-6ba8-43fa-a9b9-3fe32fe0a4b6 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:48.219777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae363cb-6ba8-43fa-a9b9-3fe32fe0a4b6 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:48.226612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7647bf-0984-48e6-985d-f9757b584fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:48.229832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7647bf-0984-48e6-985d-f9757b584fee admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:48.229832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7647bf-0984-48e6-985d-f9757b584fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:48.230051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7647bf-0984-48e6-985d-f9757b584fee admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:48.230977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7647bf-0984-48e6-985d-f9757b584fee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:48.233292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7647bf-0984-48e6-985d-f9757b584fee admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:48.254369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 850/1696] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:48.259708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a69b2a-7ff6-41d5-85ba-b6a64ffb8eca None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:48.280466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 847/1697] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:48 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:48.281867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7647bf-0984-48e6-985d-f9757b584fee admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:48.286282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a69b2a-7ff6-41d5-85ba-b6a64ffb8eca None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:48.422746 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:48.424942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5b34c-072e-4c5a-bb89-4ebd64ac6e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:48.427112 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5b34c-072e-4c5a-bb89-4ebd64ac6e82 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:48.427112 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5b34c-072e-4c5a-bb89-4ebd64ac6e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:48.427658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5b34c-072e-4c5a-bb89-4ebd64ac6e82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:48.428709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5b34c-072e-4c5a-bb89-4ebd64ac6e82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:48.434771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5b34c-072e-4c5a-bb89-4ebd64ac6e82 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:48.458970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7647bf-0984-48e6-985d-f9757b584fee admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:48.467894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89047ef-6bd7-4767-b872-766487b20d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:48.468961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89047ef-6bd7-4767-b872-766487b20d29 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:48.469076 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 851/1698] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:48 2022] POST /identity/v3/projects => generated 387 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:48.469657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89047ef-6bd7-4767-b872-766487b20d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:48.469821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89047ef-6bd7-4767-b872-766487b20d29 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:48.470561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89047ef-6bd7-4767-b872-766487b20d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:48.472777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89047ef-6bd7-4767-b872-766487b20d29 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:48.473155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5b34c-072e-4c5a-bb89-4ebd64ac6e82 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:48.503457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 848/1699] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:48 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:48.512346 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89047ef-6bd7-4767-b872-766487b20d29 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:48.535856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89047ef-6bd7-4767-b872-766487b20d29 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:48.618770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5b34c-072e-4c5a-bb89-4ebd64ac6e82 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:48.629985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c359f2-c7d2-4836-87a0-e9e2419a290a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:48.630738 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c359f2-c7d2-4836-87a0-e9e2419a290a admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:48.633034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c359f2-c7d2-4836-87a0-e9e2419a290a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:48.633260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c359f2-c7d2-4836-87a0-e9e2419a290a admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:48.633999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c359f2-c7d2-4836-87a0-e9e2419a290a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:48.640598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c359f2-c7d2-4836-87a0-e9e2419a290a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:48.669943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ddf8c-356b-4e42-ac5b-8ed5517ae1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:48.671200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ddf8c-356b-4e42-ac5b-8ed5517ae1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:48.671903 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ddf8c-356b-4e42-ac5b-8ed5517ae1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:48.674888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ddf8c-356b-4e42-ac5b-8ed5517ae1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:48.676658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ddf8c-356b-4e42-ac5b-8ed5517ae1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:48.683937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ddf8c-356b-4e42-ac5b-8ed5517ae1b3 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:48.685974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 852/1700] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:48 2022] POST /identity/v3/users => generated 300 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:19:48.689048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c359f2-c7d2-4836-87a0-e9e2419a290a admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:48.705579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c359f2-c7d2-4836-87a0-e9e2419a290a admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:48.708195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 849/1701] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:48.708195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ddf8c-356b-4e42-ac5b-8ed5517ae1b3 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:48.722028 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ddf8c-356b-4e42-ac5b-8ed5517ae1b3 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:48.727395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04774c-32cf-4c7a-9a6a-d1407a2a9822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:48.728379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04774c-32cf-4c7a-9a6a-d1407a2a9822 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:48.728739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04774c-32cf-4c7a-9a6a-d1407a2a9822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:48.729088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04774c-32cf-4c7a-9a6a-d1407a2a9822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:48.731967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83852006-945f-4f4e-868b-4219a57285ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:48.733506 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83852006-945f-4f4e-868b-4219a57285ac admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:48.733794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83852006-945f-4f4e-868b-4219a57285ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:48.734350 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83852006-945f-4f4e-868b-4219a57285ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:48.735005 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04774c-32cf-4c7a-9a6a-d1407a2a9822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:48.736058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83852006-945f-4f4e-868b-4219a57285ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:48.738928 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83852006-945f-4f4e-868b-4219a57285ac admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:48.740111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04774c-32cf-4c7a-9a6a-d1407a2a9822 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:48.756701 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 853/1702] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:48.757406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 850/1703] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:48.758467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04774c-32cf-4c7a-9a6a-d1407a2a9822 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:48.761244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83852006-945f-4f4e-868b-4219a57285ac admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:48.773850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83852006-945f-4f4e-868b-4219a57285ac admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:48.780573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04774c-32cf-4c7a-9a6a-d1407a2a9822 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:48.782184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1b87e-4e2b-4b2d-8b09-7452079a3c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:48.783405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1b87e-4e2b-4b2d-8b09-7452079a3c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:48.783878 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1b87e-4e2b-4b2d-8b09-7452079a3c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:48.784311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1b87e-4e2b-4b2d-8b09-7452079a3c16 admin admin] PATH_INFO: `/v3/projects/3ecd7923cb2d4e84bfb88342f89b8ea6/users/d672448f03be4925912ba706fa3be6dc/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:48.788830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c06074-e307-4784-a726-03d65ebba93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:48.791299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06074-e307-4784-a726-03d65ebba93b admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:48.791918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06074-e307-4784-a726-03d65ebba93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:48.792382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06074-e307-4784-a726-03d65ebba93b admin admin] PATH_INFO: `/v3/projects/3e5e2e90d8454a19a2db67bcf4eeac59/users/024f7a86883348a09eab7de3570f0b8a/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:48.808984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1b87e-4e2b-4b2d-8b09-7452079a3c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ecd7923cb2d4e84bfb88342f89b8ea6, user_id=d672448f03be4925912ba706fa3be6dc, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:48.815145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1b87e-4e2b-4b2d-8b09-7452079a3c16 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:48.831634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c06074-e307-4784-a726-03d65ebba93b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e5e2e90d8454a19a2db67bcf4eeac59, user_id=024f7a86883348a09eab7de3570f0b8a, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:48.835383 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 851/1704] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:48 2022] PUT /identity/v3/projects/3ecd7923cb2d4e84bfb88342f89b8ea6/users/d672448f03be4925912ba706fa3be6dc/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:48.837514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c06074-e307-4784-a726-03d65ebba93b admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:48.839527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1b87e-4e2b-4b2d-8b09-7452079a3c16 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:48.842596 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:48.866741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 854/1705] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:48 2022] PUT /identity/v3/projects/3e5e2e90d8454a19a2db67bcf4eeac59/users/024f7a86883348a09eab7de3570f0b8a/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:48.878216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e8eff-557d-490b-9024-d56a45ecd900 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:48.878505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e8eff-557d-490b-9024-d56a45ecd900 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:48.878786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e8eff-557d-490b-9024-d56a45ecd900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:48.911843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-266e8eff-557d-490b-9024-d56a45ecd900 None None] MFA Rules not processed for user `024f7a86883348a09eab7de3570f0b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:48.913398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-266e8eff-557d-490b-9024-d56a45ecd900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:48.966654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1b87e-4e2b-4b2d-8b09-7452079a3c16 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:49.026730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:49.055678 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 855/1706] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:49.060959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e8eff-557d-490b-9024-d56a45ecd900 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:49.070000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c7f9b-3f4a-4ab8-90a4-6d8d0ee7d93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:49.070693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c7f9b-3f4a-4ab8-90a4-6d8d0ee7d93b admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:49.070983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c7f9b-3f4a-4ab8-90a4-6d8d0ee7d93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:49.071696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c7f9b-3f4a-4ab8-90a4-6d8d0ee7d93b admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:49.073675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c7f9b-3f4a-4ab8-90a4-6d8d0ee7d93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:49.075394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c7f9b-3f4a-4ab8-90a4-6d8d0ee7d93b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:49.111200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 852/1707] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:49.117099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c7f9b-3f4a-4ab8-90a4-6d8d0ee7d93b admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:49.144601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c7f9b-3f4a-4ab8-90a4-6d8d0ee7d93b admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:49.170441 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5500c-ade7-45c1-a0fd-3bd2cb4e5c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:49.173118 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5500c-ade7-45c1-a0fd-3bd2cb4e5c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:49.174042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5500c-ade7-45c1-a0fd-3bd2cb4e5c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:49.174330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5500c-ade7-45c1-a0fd-3bd2cb4e5c49 admin admin] PATH_INFO: `/v3/projects/3ecd7923cb2d4e84bfb88342f89b8ea6/users/d672448f03be4925912ba706fa3be6dc/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:49.180236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5500c-ade7-45c1-a0fd-3bd2cb4e5c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ecd7923cb2d4e84bfb88342f89b8ea6, user_id=d672448f03be4925912ba706fa3be6dc, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:49.189770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5500c-ade7-45c1-a0fd-3bd2cb4e5c49 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:49.207765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 853/1708] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:49 2022] PUT /identity/v3/projects/3ecd7923cb2d4e84bfb88342f89b8ea6/users/d672448f03be4925912ba706fa3be6dc/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:49.208360 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e8eff-557d-490b-9024-d56a45ecd900 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:49.220763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4e84d-1e2f-4774-ae40-b1ef47fc5e74 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:49.221050 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4e84d-1e2f-4774-ae40-b1ef47fc5e74 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:49.221424 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4e84d-1e2f-4774-ae40-b1ef47fc5e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:49.259974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-52c4e84d-1e2f-4774-ae40-b1ef47fc5e74 None None] MFA Rules not processed for user `d672448f03be4925912ba706fa3be6dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:49.261695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-52c4e84d-1e2f-4774-ae40-b1ef47fc5e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:49.337331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b35f55-6941-4d60-a7f5-92cb0ebb4935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:49.338033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b35f55-6941-4d60-a7f5-92cb0ebb4935 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:49.338324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b35f55-6941-4d60-a7f5-92cb0ebb4935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:49.338609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b35f55-6941-4d60-a7f5-92cb0ebb4935 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:49.339316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b35f55-6941-4d60-a7f5-92cb0ebb4935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:49.346235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b35f55-6941-4d60-a7f5-92cb0ebb4935 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:49.381441 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 854/1709] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:49.399815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4e84d-1e2f-4774-ae40-b1ef47fc5e74 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:49.402605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 856/1710] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:49 2022] POST /identity/v3/projects => generated 377 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:49.406311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b35f55-6941-4d60-a7f5-92cb0ebb4935 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:49.460750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:49.536751 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4e84d-1e2f-4774-ae40-b1ef47fc5e74 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:49.539187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b35f55-6941-4d60-a7f5-92cb0ebb4935 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:49.544948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75679517-5a4b-4f42-bb92-336b047cff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:49.545724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75679517-5a4b-4f42-bb92-336b047cff1e admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:49.546016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75679517-5a4b-4f42-bb92-336b047cff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:49.546349 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75679517-5a4b-4f42-bb92-336b047cff1e admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:49.551240 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75679517-5a4b-4f42-bb92-336b047cff1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:49.553344 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75679517-5a4b-4f42-bb92-336b047cff1e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:49.592129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 857/1711] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:49 2022] POST /identity/v3/users => generated 295 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:49.594650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75679517-5a4b-4f42-bb92-336b047cff1e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:49.616013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75679517-5a4b-4f42-bb92-336b047cff1e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:49.628980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0daf2-f4f2-4378-a726-a1ffdaf485e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:49.629465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0daf2-f4f2-4378-a726-a1ffdaf485e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:49.630051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0daf2-f4f2-4378-a726-a1ffdaf485e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:49.630409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0daf2-f4f2-4378-a726-a1ffdaf485e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:49.631263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0daf2-f4f2-4378-a726-a1ffdaf485e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:49.635434 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0daf2-f4f2-4378-a726-a1ffdaf485e8 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:49.671257 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 858/1712] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:49.672183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0daf2-f4f2-4378-a726-a1ffdaf485e8 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:49.689010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0daf2-f4f2-4378-a726-a1ffdaf485e8 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:49.701933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766ec91-0a9b-4526-aa2a-fce0cbbd8115 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:49.703165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766ec91-0a9b-4526-aa2a-fce0cbbd8115 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:49.703285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d00428-fe03-4d8c-bac4-24481838e88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:49.703668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766ec91-0a9b-4526-aa2a-fce0cbbd8115 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:49.704316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d00428-fe03-4d8c-bac4-24481838e88a admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:49.704432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766ec91-0a9b-4526-aa2a-fce0cbbd8115 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:49.705001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d00428-fe03-4d8c-bac4-24481838e88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:49.705407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d00428-fe03-4d8c-bac4-24481838e88a admin admin] PATH_INFO: `/v3/projects/a9496f21d58140b59861774da9e944d6/users/54e9add447ae404e95387bffd87b2abb/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:49.739929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:49.754344 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d00428-fe03-4d8c-bac4-24481838e88a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9496f21d58140b59861774da9e944d6, user_id=54e9add447ae404e95387bffd87b2abb, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:49.757841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d00428-fe03-4d8c-bac4-24481838e88a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:49.791561 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 859/1713] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:49 2022] PUT /identity/v3/projects/a9496f21d58140b59861774da9e944d6/users/54e9add447ae404e95387bffd87b2abb/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:49.794175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d00428-fe03-4d8c-bac4-24481838e88a admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:49.834461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766ec91-0a9b-4526-aa2a-fce0cbbd8115 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:49.838835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766ec91-0a9b-4526-aa2a-fce0cbbd8115 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:49.871177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 855/1714] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:49.923224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d00428-fe03-4d8c-bac4-24481838e88a admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:49.939214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0d86c-5bf1-4a20-a46b-ddaeae33ef19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:49.942235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0d86c-5bf1-4a20-a46b-ddaeae33ef19 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:49.970323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0d86c-5bf1-4a20-a46b-ddaeae33ef19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:49.972137 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0d86c-5bf1-4a20-a46b-ddaeae33ef19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:49.976103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0d86c-5bf1-4a20-a46b-ddaeae33ef19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:49.980593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0d86c-5bf1-4a20-a46b-ddaeae33ef19 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:50.007355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 860/1715] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:50.011884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766ec91-0a9b-4526-aa2a-fce0cbbd8115 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:50.027403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766ec91-0a9b-4526-aa2a-fce0cbbd8115 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:50.033159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b518e-8784-422a-bbf9-698721c602ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:50.034346 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b518e-8784-422a-bbf9-698721c602ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:50.034984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b518e-8784-422a-bbf9-698721c602ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:50.035456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b518e-8784-422a-bbf9-698721c602ab admin admin] PATH_INFO: `/v3/projects/a9496f21d58140b59861774da9e944d6/users/54e9add447ae404e95387bffd87b2abb/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:50.039771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b518e-8784-422a-bbf9-698721c602ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9496f21d58140b59861774da9e944d6, user_id=54e9add447ae404e95387bffd87b2abb, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:50.043764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b518e-8784-422a-bbf9-698721c602ab admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:50.062610 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 856/1716] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:50 2022] PUT /identity/v3/projects/a9496f21d58140b59861774da9e944d6/users/54e9add447ae404e95387bffd87b2abb/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:50.073150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6441a0-222b-461e-9c8b-20a3eee3f20d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:50.073447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6441a0-222b-461e-9c8b-20a3eee3f20d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:50.073731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6441a0-222b-461e-9c8b-20a3eee3f20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:50.126726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b518e-8784-422a-bbf9-698721c602ab admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:50.133986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-1f6441a0-222b-461e-9c8b-20a3eee3f20d None None] MFA Rules not processed for user `54e9add447ae404e95387bffd87b2abb`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:50.136206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-1f6441a0-222b-461e-9c8b-20a3eee3f20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:50.240737 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b518e-8784-422a-bbf9-698721c602ab admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:50.246821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b147979-02c1-400b-b684-545be57d1cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:50.248036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b147979-02c1-400b-b684-545be57d1cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:50.248492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b147979-02c1-400b-b684-545be57d1cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:50.248966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b147979-02c1-400b-b684-545be57d1cab admin admin] PATH_INFO: `/v3/users/a343b5e63854409681f9b4d8c7f70836` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:50.250351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b147979-02c1-400b-b684-545be57d1cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a343b5e63854409681f9b4d8c7f70836)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:50.252696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b147979-02c1-400b-b684-545be57d1cab admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:50.258934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:50.281513 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 861/1717] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:50.295955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6441a0-222b-461e-9c8b-20a3eee3f20d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:50.367901 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6b147979-02c1-400b-b684-545be57d1cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a343b5e63854409681f9b4d8c7f70836'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:50.393953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6b147979-02c1-400b-b684-545be57d1cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a343b5e63854409681f9b4d8c7f70836'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:50.398670 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6b147979-02c1-400b-b684-545be57d1cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a343b5e63854409681f9b4d8c7f70836'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:50.455601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6b147979-02c1-400b-b684-545be57d1cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a343b5e63854409681f9b4d8c7f70836'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:50.458954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6b147979-02c1-400b-b684-545be57d1cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a343b5e63854409681f9b4d8c7f70836'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:50.503468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6441a0-222b-461e-9c8b-20a3eee3f20d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:50.508890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 857/1718] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:19:50 2022] DELETE /identity/v3/users/a343b5e63854409681f9b4d8c7f70836 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:50.514353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1311164-cdf2-4a7f-b12f-894edcd94d19 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:50.515125 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1311164-cdf2-4a7f-b12f-894edcd94d19 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:50.516217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1311164-cdf2-4a7f-b12f-894edcd94d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:50.517002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd31d8-8d4b-46fc-93eb-a0cf8ad6d288 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:50.517914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd31d8-8d4b-46fc-93eb-a0cf8ad6d288 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:50.519545 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd31d8-8d4b-46fc-93eb-a0cf8ad6d288 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:50.519900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd31d8-8d4b-46fc-93eb-a0cf8ad6d288 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:50.530775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e4fd31d8-8d4b-46fc-93eb-a0cf8ad6d288 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:50.572804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-b1311164-cdf2-4a7f-b12f-894edcd94d19 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:50.574449 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd31d8-8d4b-46fc-93eb-a0cf8ad6d288 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:50.576213 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd31d8-8d4b-46fc-93eb-a0cf8ad6d288 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:50.589062 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b1311164-cdf2-4a7f-b12f-894edcd94d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:50.621704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 862/1719] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:50.643146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:50.711742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd31d8-8d4b-46fc-93eb-a0cf8ad6d288 service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:50.713269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e4fd31d8-8d4b-46fc-93eb-a0cf8ad6d288 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:50.736868 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd31d8-8d4b-46fc-93eb-a0cf8ad6d288 service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:50.792116 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 858/1720] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:50.856332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1311164-cdf2-4a7f-b12f-894edcd94d19 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:50.857511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4f5ad-f0a1-468b-9094-8bd21646e07e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:50.858196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4f5ad-f0a1-468b-9094-8bd21646e07e service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:50.858490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4f5ad-f0a1-468b-9094-8bd21646e07e service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:50.858624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b1311164-cdf2-4a7f-b12f-894edcd94d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:50.858897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4f5ad-f0a1-468b-9094-8bd21646e07e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:50.861052 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d1f4f5ad-f0a1-468b-9094-8bd21646e07e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:50.958271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4f5ad-f0a1-468b-9094-8bd21646e07e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:50.959896 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4f5ad-f0a1-468b-9094-8bd21646e07e service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:50.977001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 863/1721] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:19:50.987466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1311164-cdf2-4a7f-b12f-894edcd94d19 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:51.016615 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bd90a-1016-476a-923a-9ce1e489ed22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:51.017313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bd90a-1016-476a-923a-9ce1e489ed22 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:51.017603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bd90a-1016-476a-923a-9ce1e489ed22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:51.017890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bd90a-1016-476a-923a-9ce1e489ed22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:51.069092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bd90a-1016-476a-923a-9ce1e489ed22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:51.070573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bd90a-1016-476a-923a-9ce1e489ed22 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:51.101842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 859/1722] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:51.875890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b553ce-8fd4-441a-9c9d-0eb6c9a56658 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:51.876750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b553ce-8fd4-441a-9c9d-0eb6c9a56658 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:51.877181 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b553ce-8fd4-441a-9c9d-0eb6c9a56658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:51.934921 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-d2b553ce-8fd4-441a-9c9d-0eb6c9a56658 None None] MFA Rules not processed for user `b3ca63f8f8484ddf9d7c72749ba7f482`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:51.942617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d2b553ce-8fd4-441a-9c9d-0eb6c9a56658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:52.018785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:52.047302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 864/1723] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:52.669071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bd90a-1016-476a-923a-9ce1e489ed22 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:52.670434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-071bd90a-1016-476a-923a-9ce1e489ed22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:52.695964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bd90a-1016-476a-923a-9ce1e489ed22 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:52.716427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d1d2b-5dee-47a3-93db-907bb452c852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:52.717001 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d1d2b-5dee-47a3-93db-907bb452c852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:52.717304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d1d2b-5dee-47a3-93db-907bb452c852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:52.717568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d1d2b-5dee-47a3-93db-907bb452c852 admin admin] PATH_INFO: `/v3/projects/41cc777c1a32431087757206d5f27e6b` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:52.724602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d1d2b-5dee-47a3-93db-907bb452c852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41cc777c1a32431087757206d5f27e6b)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:52.726592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d1d2b-5dee-47a3-93db-907bb452c852 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:52.879401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-335d1d2b-5dee-47a3-93db-907bb452c852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41cc777c1a32431087757206d5f27e6b', 'request_id': 'req-335d1d2b-5dee-47a3-93db-907bb452c852'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:52.890519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-335d1d2b-5dee-47a3-93db-907bb452c852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41cc777c1a32431087757206d5f27e6b', 'request_id': 'req-335d1d2b-5dee-47a3-93db-907bb452c852'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:52.904713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-335d1d2b-5dee-47a3-93db-907bb452c852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41cc777c1a32431087757206d5f27e6b', 'request_id': 'req-335d1d2b-5dee-47a3-93db-907bb452c852'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:52.932529 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-335d1d2b-5dee-47a3-93db-907bb452c852 admin admin] The token cache is being invalidate because project 41cc777c1a32431087757206d5f27e6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:19:52.932883 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-335d1d2b-5dee-47a3-93db-907bb452c852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:52.935121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-335d1d2b-5dee-47a3-93db-907bb452c852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:52.938472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 860/1724] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:19:52 2022] DELETE /identity/v3/projects/41cc777c1a32431087757206d5f27e6b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:52.942793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b553ce-8fd4-441a-9c9d-0eb6c9a56658 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:52.992877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b553ce-8fd4-441a-9c9d-0eb6c9a56658 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:53.206555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:53.208006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:53.211197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:53.217560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] PATH_INFO: `/v3/users/d672448f03be4925912ba706fa3be6dc` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:53.217560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d672448f03be4925912ba706fa3be6dc)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:53.246808 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:53.379901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd672448f03be4925912ba706fa3be6dc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:53.381834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd672448f03be4925912ba706fa3be6dc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:53.423020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd672448f03be4925912ba706fa3be6dc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:53.463101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd672448f03be4925912ba706fa3be6dc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:53.463746 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd672448f03be4925912ba706fa3be6dc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:53.483329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe8af8-8fee-411e-9d6b-6ed8a82f5fe8 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:53.483626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe8af8-8fee-411e-9d6b-6ed8a82f5fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:53.483915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe8af8-8fee-411e-9d6b-6ed8a82f5fe8 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:53.485684 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 861/1725] 10.222.0.33 () {54 vars in 912 bytes} [Fri Jun 17 07:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:19:53.492711 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 865/1726] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:19:52 2022] DELETE /identity/v3/users/d672448f03be4925912ba706fa3be6dc => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:53.492856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c037d65-f1fa-4999-899d-fa08cbcb26a6 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:53.492856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c037d65-f1fa-4999-899d-fa08cbcb26a6 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:53.492856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c037d65-f1fa-4999-899d-fa08cbcb26a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:53.570381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-9c037d65-f1fa-4999-899d-fa08cbcb26a6 None None] MFA Rules not processed for user `7e0797679c07454abc867cd7b30c8b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:53.571382 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-9c037d65-f1fa-4999-899d-fa08cbcb26a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:53.642098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:53.693209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 862/1727] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:19:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 17 07:19:53.715164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:53.717164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:53.762188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692d873-6b64-40b7-8886-62089db0d997 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:53.776216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb05547-8494-4bf5-8e8d-bff1ca9e3c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:53.778712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb05547-8494-4bf5-8e8d-bff1ca9e3c6e service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:53.779010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb05547-8494-4bf5-8e8d-bff1ca9e3c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:53.779291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb05547-8494-4bf5-8e8d-bff1ca9e3c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:53.802209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb05547-8494-4bf5-8e8d-bff1ca9e3c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:53.805120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb05547-8494-4bf5-8e8d-bff1ca9e3c6e service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:53.840742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 866/1728] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:55.107683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2690c-e7d1-4db0-aab2-a2f51411de1c None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:55.108202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2690c-e7d1-4db0-aab2-a2f51411de1c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:55.108311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2690c-e7d1-4db0-aab2-a2f51411de1c None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:55.109857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 863/1729] 10.222.0.33 () {54 vars in 960 bytes} [Fri Jun 17 07:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:19:55.117474 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53326978-ed4b-4999-b432-6002cfb50069 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:55.120589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53326978-ed4b-4999-b432-6002cfb50069 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:55.121083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53326978-ed4b-4999-b432-6002cfb50069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:55.144319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-53326978-ed4b-4999-b432-6002cfb50069 None None] MFA Rules not processed for user `b8cf02a52d354517b363c0aaf477550c`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:55.145832 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-53326978-ed4b-4999-b432-6002cfb50069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:55.260923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:55.321669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2690c-e7d1-4db0-aab2-a2f51411de1c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:55.322231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 867/1730] 10.222.0.33 () {58 vars in 1057 bytes} [Fri Jun 17 07:19:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:55.360849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2690c-e7d1-4db0-aab2-a2f51411de1c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:55.372312 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee30c8a-746b-4f8a-978d-c635eb735b1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8cf02a52d354517b363c0aaf477550c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:55.373150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee30c8a-746b-4f8a-978d-c635eb735b1b service placement] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:55.373576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee30c8a-746b-4f8a-978d-c635eb735b1b service placement] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:55.373956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee30c8a-746b-4f8a-978d-c635eb735b1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:55.376025 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3ee30c8a-746b-4f8a-978d-c635eb735b1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:55.392591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee30c8a-746b-4f8a-978d-c635eb735b1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:55.400151 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee30c8a-746b-4f8a-978d-c635eb735b1b service placement] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:55.417461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 864/1731] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:55.660544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53326978-ed4b-4999-b432-6002cfb50069 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:55.663085 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-53326978-ed4b-4999-b432-6002cfb50069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:55.745056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63869e66-ead1-496f-8208-a649a26bd13c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:55.745558 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63869e66-ead1-496f-8208-a649a26bd13c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:55.746981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63869e66-ead1-496f-8208-a649a26bd13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:55.866430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-63869e66-ead1-496f-8208-a649a26bd13c None None] MFA Rules not processed for user `31e94cc1c6a743ce898424ad7ba26cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:55.870832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-63869e66-ead1-496f-8208-a649a26bd13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:55.883526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53326978-ed4b-4999-b432-6002cfb50069 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:55.907289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:55.918413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:55.918413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:55.919003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d admin admin] PATH_INFO: `/v3/projects/3ecd7923cb2d4e84bfb88342f89b8ea6` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:55.920871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ecd7923cb2d4e84bfb88342f89b8ea6)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:55.923307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:56.090884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ecd7923cb2d4e84bfb88342f89b8ea6', 'request_id': 'req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:56.123661 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ecd7923cb2d4e84bfb88342f89b8ea6', 'request_id': 'req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:56.147094 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 865/1732] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:56.156199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ecd7923cb2d4e84bfb88342f89b8ea6', 'request_id': 'req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:56.170637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63869e66-ead1-496f-8208-a649a26bd13c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:56.173310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-63869e66-ead1-496f-8208-a649a26bd13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:56.206095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d admin admin] The token cache is being invalidate because project 3ecd7923cb2d4e84bfb88342f89b8ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:19:56.208728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:56.209606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c9414ff8-3f10-4c6d-abe4-8b81e78c2b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:19:56.237195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 868/1733] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:19:55 2022] DELETE /identity/v3/projects/3ecd7923cb2d4e84bfb88342f89b8ea6 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:56.241308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50a202-b15b-46c1-a4e9-e7582236a5ea None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:56.241607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50a202-b15b-46c1-a4e9-e7582236a5ea None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:56.241890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50a202-b15b-46c1-a4e9-e7582236a5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:56.305123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-9f50a202-b15b-46c1-a4e9-e7582236a5ea None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:56.306998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9f50a202-b15b-46c1-a4e9-e7582236a5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:56.455425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63869e66-ead1-496f-8208-a649a26bd13c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:56.501698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 869/1734] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:56.516682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e470ede-d937-4f2f-a50d-fe114e8e44da None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:56.517191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e470ede-d937-4f2f-a50d-fe114e8e44da None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:56.517680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e470ede-d937-4f2f-a50d-fe114e8e44da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:56.572949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:56.583905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-7e470ede-d937-4f2f-a50d-fe114e8e44da None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:56.585305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7e470ede-d937-4f2f-a50d-fe114e8e44da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:56.606317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 870/1735] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:56.606492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a73ee9-764b-498f-a43e-7a8f9ea77a13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:56.607554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e470ede-d937-4f2f-a50d-fe114e8e44da None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:56.608668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a73ee9-764b-498f-a43e-7a8f9ea77a13 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:56.608993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a73ee9-764b-498f-a43e-7a8f9ea77a13 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:56.610952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a73ee9-764b-498f-a43e-7a8f9ea77a13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:56.620001 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-15a73ee9-764b-498f-a43e-7a8f9ea77a13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:56.631126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e470ede-d937-4f2f-a50d-fe114e8e44da None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:56.645718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f790d-94f0-4c0c-9ff4-13210007153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:56.646478 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f790d-94f0-4c0c-9ff4-13210007153e admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:56.646832 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f790d-94f0-4c0c-9ff4-13210007153e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:56.647177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f790d-94f0-4c0c-9ff4-13210007153e admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:56.648054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f790d-94f0-4c0c-9ff4-13210007153e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:19:56.648455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f790d-94f0-4c0c-9ff4-13210007153e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:56.650863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f790d-94f0-4c0c-9ff4-13210007153e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:56.679478 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 871/1736] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:19:56.684283 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f790d-94f0-4c0c-9ff4-13210007153e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:56.710173 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f790d-94f0-4c0c-9ff4-13210007153e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:56.722289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb033bea-e718-4317-9461-c5279823ca43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:56.723664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb033bea-e718-4317-9461-c5279823ca43 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:56.726538 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb033bea-e718-4317-9461-c5279823ca43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:56.727083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb033bea-e718-4317-9461-c5279823ca43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:56.727958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb033bea-e718-4317-9461-c5279823ca43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:56.730169 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb033bea-e718-4317-9461-c5279823ca43 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:56.741935 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:56.777387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 872/1737] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:56 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:19:56.779096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303250a9-8be1-415d-89cf-25b8c3ebf42c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:56.779496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303250a9-8be1-415d-89cf-25b8c3ebf42c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:56.779902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303250a9-8be1-415d-89cf-25b8c3ebf42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:56.784341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a73ee9-764b-498f-a43e-7a8f9ea77a13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:56.786903 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a73ee9-764b-498f-a43e-7a8f9ea77a13 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:56.807489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 866/1738] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:56.811073 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a73ee9-764b-498f-a43e-7a8f9ea77a13 service nova] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:56.817890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-303250a9-8be1-415d-89cf-25b8c3ebf42c None None] MFA Rules not processed for user `c11f44eb67cd4923a840171833bb572a`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:56.819348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-303250a9-8be1-415d-89cf-25b8c3ebf42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:56.833235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a73ee9-764b-498f-a43e-7a8f9ea77a13 service nova] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:56.937071 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 873/1739] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:56.950829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6141999-fc7d-485d-85fa-75a3b0102191 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:56.951411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6141999-fc7d-485d-85fa-75a3b0102191 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:56.952145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6141999-fc7d-485d-85fa-75a3b0102191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:56.964244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43060ec6-7249-4f9a-9e89-32135243500b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:56.965450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43060ec6-7249-4f9a-9e89-32135243500b admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:56.965450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43060ec6-7249-4f9a-9e89-32135243500b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:56.965675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43060ec6-7249-4f9a-9e89-32135243500b admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:56.966680 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43060ec6-7249-4f9a-9e89-32135243500b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:56.968526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43060ec6-7249-4f9a-9e89-32135243500b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:56.996682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-a6141999-fc7d-485d-85fa-75a3b0102191 None None] MFA Rules not processed for user `c11f44eb67cd4923a840171833bb572a`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:56.998294 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a6141999-fc7d-485d-85fa-75a3b0102191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:57.005608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 867/1740] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:56 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:57.010725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43060ec6-7249-4f9a-9e89-32135243500b admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:57.017798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 874/1741] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:57.030183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6141999-fc7d-485d-85fa-75a3b0102191 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:57.032290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a6141999-fc7d-485d-85fa-75a3b0102191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:57.033660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43060ec6-7249-4f9a-9e89-32135243500b admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:57.039270 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0e276-882a-4619-bf5d-56c858cfa421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:57.039828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0e276-882a-4619-bf5d-56c858cfa421 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:57.040219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0e276-882a-4619-bf5d-56c858cfa421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:57.040514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0e276-882a-4619-bf5d-56c858cfa421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:57.041317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0e276-882a-4619-bf5d-56c858cfa421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:57.042949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0e276-882a-4619-bf5d-56c858cfa421 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:57.063441 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 868/1742] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:19:57.064502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0e276-882a-4619-bf5d-56c858cfa421 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:57.078590 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0e276-882a-4619-bf5d-56c858cfa421 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:57.091256 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bc7e0-0d9d-45db-af12-240c026078c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:57.092109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bc7e0-0d9d-45db-af12-240c026078c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:57.092382 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bc7e0-0d9d-45db-af12-240c026078c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:57.092666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bc7e0-0d9d-45db-af12-240c026078c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:57.093740 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88bc7e0-0d9d-45db-af12-240c026078c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:57.095623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88bc7e0-0d9d-45db-af12-240c026078c7 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:57.126047 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 869/1743] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:57.127360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bc7e0-0d9d-45db-af12-240c026078c7 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:57.144643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bc7e0-0d9d-45db-af12-240c026078c7 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:57.150699 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5ad0a-42b1-4dc3-a90f-ca9fc668b8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:57.152777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5ad0a-42b1-4dc3-a90f-ca9fc668b8e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:57.153222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5ad0a-42b1-4dc3-a90f-ca9fc668b8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:57.153654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5ad0a-42b1-4dc3-a90f-ca9fc668b8e7 admin admin] PATH_INFO: `/v3/projects/6c67b87260f4469c9d244dafa43b8857/users/ed56fb506b4144059c28f5802a73f28c/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:57.227979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5ad0a-42b1-4dc3-a90f-ca9fc668b8e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c67b87260f4469c9d244dafa43b8857, user_id=ed56fb506b4144059c28f5802a73f28c, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:57.233618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5ad0a-42b1-4dc3-a90f-ca9fc668b8e7 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:57.246580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6141999-fc7d-485d-85fa-75a3b0102191 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:57.266750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 870/1744] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:57 2022] PUT /identity/v3/projects/6c67b87260f4469c9d244dafa43b8857/users/ed56fb506b4144059c28f5802a73f28c/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:57.284398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b6226-cc86-4206-9177-50a756a4c3da None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:57.284745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b6226-cc86-4206-9177-50a756a4c3da None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:57.291307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b6226-cc86-4206-9177-50a756a4c3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:57.415046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-d83b6226-cc86-4206-9177-50a756a4c3da None None] MFA Rules not processed for user `ed56fb506b4144059c28f5802a73f28c`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:57.416541 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d83b6226-cc86-4206-9177-50a756a4c3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:57.499281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:57.515198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f17348-7bd9-4e23-ad28-7c1606393729 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:57.516125 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f17348-7bd9-4e23-ad28-7c1606393729 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:57.516781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f17348-7bd9-4e23-ad28-7c1606393729 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:57.527105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f17348-7bd9-4e23-ad28-7c1606393729 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:57.542225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:57.601248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 871/1745] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:57.614380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b6226-cc86-4206-9177-50a756a4c3da None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:57.680047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f17348-7bd9-4e23-ad28-7c1606393729 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:57.691998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f17348-7bd9-4e23-ad28-7c1606393729 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:57.707495 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 875/1746] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:57.709694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55012253-c631-488f-9970-5e43f1be6a1d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:57.710023 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55012253-c631-488f-9970-5e43f1be6a1d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:57.710847 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55012253-c631-488f-9970-5e43f1be6a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:57.786519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b6226-cc86-4206-9177-50a756a4c3da None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:57.791025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-55012253-c631-488f-9970-5e43f1be6a1d None None] MFA Rules not processed for user `6d04750a83fb465e8fba24c1143d4eff`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:57.792431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-55012253-c631-488f-9970-5e43f1be6a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:57.800584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b15c6-8495-4632-8cb8-194c7b567620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:57.804044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b15c6-8495-4632-8cb8-194c7b567620 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:57.804543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b15c6-8495-4632-8cb8-194c7b567620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:57.804980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b15c6-8495-4632-8cb8-194c7b567620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:57.806930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b15c6-8495-4632-8cb8-194c7b567620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:57.814916 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b15c6-8495-4632-8cb8-194c7b567620 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:57.903067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 872/1747] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:19:57 2022] POST /identity/v3/projects => generated 381 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:57.907608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b15c6-8495-4632-8cb8-194c7b567620 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:58.033538 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 876/1748] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:58.051670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922be51-a652-46fe-883c-f6bf1538990b None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:58.051966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922be51-a652-46fe-883c-f6bf1538990b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:58.052384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922be51-a652-46fe-883c-f6bf1538990b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:58.085131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-7922be51-a652-46fe-883c-f6bf1538990b None None] MFA Rules not processed for user `6d04750a83fb465e8fba24c1143d4eff`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:58.087415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7922be51-a652-46fe-883c-f6bf1538990b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:58.129978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b15c6-8495-4632-8cb8-194c7b567620 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:58.146621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf917ea4-ce7a-4958-98d8-c1b9e6bcad08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:58.146621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf917ea4-ce7a-4958-98d8-c1b9e6bcad08 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:58.169426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf917ea4-ce7a-4958-98d8-c1b9e6bcad08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:58.170471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf917ea4-ce7a-4958-98d8-c1b9e6bcad08 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:58.171624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf917ea4-ce7a-4958-98d8-c1b9e6bcad08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:58.174106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf917ea4-ce7a-4958-98d8-c1b9e6bcad08 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:58.229989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 873/1749] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:19:57 2022] POST /identity/v3/users => generated 297 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:19:58.231009 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf917ea4-ce7a-4958-98d8-c1b9e6bcad08 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:58.250961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf917ea4-ce7a-4958-98d8-c1b9e6bcad08 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:58.259193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec89cd-c871-4c82-bf9d-508e174a696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:58.260838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 877/1750] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:58.262620 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec89cd-c871-4c82-bf9d-508e174a696e admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:58.263132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec89cd-c871-4c82-bf9d-508e174a696e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:58.263996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec89cd-c871-4c82-bf9d-508e174a696e admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:58.274997 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922be51-a652-46fe-883c-f6bf1538990b None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:58.276577 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec89cd-c871-4c82-bf9d-508e174a696e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:58.277950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7922be51-a652-46fe-883c-f6bf1538990b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:58.279299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec89cd-c871-4c82-bf9d-508e174a696e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:58.310391 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 874/1751] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:58.312004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec89cd-c871-4c82-bf9d-508e174a696e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:58.329476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec89cd-c871-4c82-bf9d-508e174a696e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:58.352058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2247b-a0bb-44f0-bfec-c4e635bbe86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:58.354091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2247b-a0bb-44f0-bfec-c4e635bbe86f admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:58.354654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2247b-a0bb-44f0-bfec-c4e635bbe86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:58.355043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2247b-a0bb-44f0-bfec-c4e635bbe86f admin admin] PATH_INFO: `/v3/projects/b0e7a45ffa5449539fbe11539064ffb0/users/4b9d9d836cf04984b92842d150c1d382/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:58.376407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:58.419569 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2247b-a0bb-44f0-bfec-c4e635bbe86f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e7a45ffa5449539fbe11539064ffb0, user_id=4b9d9d836cf04984b92842d150c1d382, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:58.430200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2247b-a0bb-44f0-bfec-c4e635bbe86f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:58.450868 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922be51-a652-46fe-883c-f6bf1538990b None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:58.469344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 875/1752] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:58 2022] PUT /identity/v3/projects/b0e7a45ffa5449539fbe11539064ffb0/users/4b9d9d836cf04984b92842d150c1d382/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:19:58.479015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2247b-a0bb-44f0-bfec-c4e635bbe86f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:58.507609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:58.627834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a56af4-baf7-44ff-88dd-1b75c13bca6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:58.628606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a56af4-baf7-44ff-88dd-1b75c13bca6e service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:58.628963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a56af4-baf7-44ff-88dd-1b75c13bca6e service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:58.629311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a56af4-baf7-44ff-88dd-1b75c13bca6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:58.675473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2247b-a0bb-44f0-bfec-c4e635bbe86f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:58.691016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbb515-6bee-43d2-922e-c966fa36a9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:58.691919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbb515-6bee-43d2-922e-c966fa36a9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:58.692248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbb515-6bee-43d2-922e-c966fa36a9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:58.692537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbb515-6bee-43d2-922e-c966fa36a9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:58.693438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbb515-6bee-43d2-922e-c966fa36a9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:58.695124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbb515-6bee-43d2-922e-c966fa36a9ca admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:58.724002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 876/1753] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:19:58.728361 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbb515-6bee-43d2-922e-c966fa36a9ca admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:58.741777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbb515-6bee-43d2-922e-c966fa36a9ca admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:58.748055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8dc6e-7d08-4252-943c-75206642c78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:58.749016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8dc6e-7d08-4252-943c-75206642c78b admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:58.749305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8dc6e-7d08-4252-943c-75206642c78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:58.749649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8dc6e-7d08-4252-943c-75206642c78b admin admin] PATH_INFO: `/v3/projects/b0e7a45ffa5449539fbe11539064ffb0/users/4b9d9d836cf04984b92842d150c1d382/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:58.751757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8dc6e-7d08-4252-943c-75206642c78b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e7a45ffa5449539fbe11539064ffb0, user_id=4b9d9d836cf04984b92842d150c1d382, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:58.754521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a56af4-baf7-44ff-88dd-1b75c13bca6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:58.755591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8dc6e-7d08-4252-943c-75206642c78b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:58.756178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a56af4-baf7-44ff-88dd-1b75c13bca6e service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:58.768230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 877/1754] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:19:58 2022] PUT /identity/v3/projects/b0e7a45ffa5449539fbe11539064ffb0/users/4b9d9d836cf04984b92842d150c1d382/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:19:58.779562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 878/1755] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:19:58.781450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc946f7f-b51a-475a-9e28-095c8b7b6570 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:58.782063 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc946f7f-b51a-475a-9e28-095c8b7b6570 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:58.782524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc946f7f-b51a-475a-9e28-095c8b7b6570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:58.813877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3c0fb-8091-4a66-8e0d-2a3afd8ec8a4 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:58.814113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3c0fb-8091-4a66-8e0d-2a3afd8ec8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:58.814395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3c0fb-8091-4a66-8e0d-2a3afd8ec8a4 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:58.816787 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 879/1756] 10.222.0.33 () {54 vars in 912 bytes} [Fri Jun 17 07:19:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 07:19:58.822477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbd306-e10f-4884-ba9f-f9284411f4e2 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:58.822920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbd306-e10f-4884-ba9f-f9284411f4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:58.823688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbd306-e10f-4884-ba9f-f9284411f4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:58.849199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-66cbd306-e10f-4884-ba9f-f9284411f4e2 None None] MFA Rules not processed for user `7e0797679c07454abc867cd7b30c8b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:58.853584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-66cbd306-e10f-4884-ba9f-f9284411f4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:58.866918 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-bc946f7f-b51a-475a-9e28-095c8b7b6570 None None] MFA Rules not processed for user `4b9d9d836cf04984b92842d150c1d382`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:19:58.868372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-bc946f7f-b51a-475a-9e28-095c8b7b6570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:58.938107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 880/1757] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:19:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:19:58.967855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbd306-e10f-4884-ba9f-f9284411f4e2 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:58.969421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-66cbd306-e10f-4884-ba9f-f9284411f4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:58.978059 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 878/1758] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:19:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:19:58.985550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbd306-e10f-4884-ba9f-f9284411f4e2 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:58.988353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc946f7f-b51a-475a-9e28-095c8b7b6570 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:58.990074 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf3f10-0123-4e13-860e-57b7a7e704f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:58.990442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-bc946f7f-b51a-475a-9e28-095c8b7b6570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:19:58.990878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf3f10-0123-4e13-860e-57b7a7e704f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:58.991242 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf3f10-0123-4e13-860e-57b7a7e704f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:58.991591 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf3f10-0123-4e13-860e-57b7a7e704f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:59.002559 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf3f10-0123-4e13-860e-57b7a7e704f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:59.006278 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf3f10-0123-4e13-860e-57b7a7e704f7 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:59.019706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 881/1759] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:19:59.061010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc946f7f-b51a-475a-9e28-095c8b7b6570 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:59.065325 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d2564-f2f6-4877-8e28-36307de607e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:59.066205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d2564-f2f6-4877-8e28-36307de607e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:59.066673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d2564-f2f6-4877-8e28-36307de607e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:59.067142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d2564-f2f6-4877-8e28-36307de607e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:59.079053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d2564-f2f6-4877-8e28-36307de607e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:59.089052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d2564-f2f6-4877-8e28-36307de607e8 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:59.115168 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 879/1760] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:19:59.289381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf3f10-0123-4e13-860e-57b7a7e704f7 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:19:59.299485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf3f10-0123-4e13-860e-57b7a7e704f7 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:19:59.313439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10773b3-e9bb-44e9-8c22-c645642c6261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:19:59.314240 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10773b3-e9bb-44e9-8c22-c645642c6261 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:19:59.314648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10773b3-e9bb-44e9-8c22-c645642c6261 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:19:59.325354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10773b3-e9bb-44e9-8c22-c645642c6261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:19:59.365689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:19:59.486363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10773b3-e9bb-44e9-8c22-c645642c6261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:19:59.488307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10773b3-e9bb-44e9-8c22-c645642c6261 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:19:59.506697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 882/1761] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:19:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:00.549417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571b6ef-c956-426c-a502-7ab915c5eff3 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:00.549892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571b6ef-c956-426c-a502-7ab915c5eff3 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:00.550023 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571b6ef-c956-426c-a502-7ab915c5eff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:00.590220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6571b6ef-c956-426c-a502-7ab915c5eff3 None None] MFA Rules not processed for user `c11f44eb67cd4923a840171833bb572a`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:00.611600 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6571b6ef-c956-426c-a502-7ab915c5eff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:00.666623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:00.794261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 880/1762] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:00.989431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10773b3-e9bb-44e9-8c22-c645642c6261 service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:00.991516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d10773b3-e9bb-44e9-8c22-c645642c6261 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:01.046454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeeb94a-95b5-4189-b5be-1a97e1585207 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:01.046757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeeb94a-95b5-4189-b5be-1a97e1585207 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:01.047046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeeb94a-95b5-4189-b5be-1a97e1585207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:01.133504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-bbeeb94a-95b5-4189-b5be-1a97e1585207 None None] MFA Rules not processed for user `31e94cc1c6a743ce898424ad7ba26cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:01.135431 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-bbeeb94a-95b5-4189-b5be-1a97e1585207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:01.385790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10773b3-e9bb-44e9-8c22-c645642c6261 service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:01.409492 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:01.411506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:01.411803 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:01.412109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] PATH_INFO: `/v3/users/b41a1927cdd64ee88cb7974ef48d07bd` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:01.430371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41a1927cdd64ee88cb7974ef48d07bd)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:01.430371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:01.439895 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 881/1763] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:20:01.445065 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e0839-bcee-4b98-8583-ade912f13f23 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:01.445376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e0839-bcee-4b98-8583-ade912f13f23 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:01.445665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e0839-bcee-4b98-8583-ade912f13f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:01.511335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-8b1e0839-bcee-4b98-8583-ade912f13f23 None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:01.512620 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8b1e0839-bcee-4b98-8583-ade912f13f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:01.589799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41a1927cdd64ee88cb7974ef48d07bd'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:01.590748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41a1927cdd64ee88cb7974ef48d07bd'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:01.634584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41a1927cdd64ee88cb7974ef48d07bd'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:01.671873 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 882/1764] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:20:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:01.708260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41a1927cdd64ee88cb7974ef48d07bd'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:01.710760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41a1927cdd64ee88cb7974ef48d07bd'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:01.772486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 883/1765] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:00 2022] DELETE /identity/v3/users/b41a1927cdd64ee88cb7974ef48d07bd => generated 0 bytes in 783 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:01.776025 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e675b-2ef2-4aea-b27b-f9b316e4f617 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:01.776342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e675b-2ef2-4aea-b27b-f9b316e4f617 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:01.776926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e675b-2ef2-4aea-b27b-f9b316e4f617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:01.830033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-3f3e675b-2ef2-4aea-b27b-f9b316e4f617 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:01.833558 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3f3e675b-2ef2-4aea-b27b-f9b316e4f617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:01.930930 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:01.932809 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:01.967260 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 883/1766] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:01.994937 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:02.021335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e675b-2ef2-4aea-b27b-f9b316e4f617 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:02.023481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3f3e675b-2ef2-4aea-b27b-f9b316e4f617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:02.070313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:02.107446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55e775-dc85-4498-b53c-f30745f1dd2d admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:02.122990 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39134d-1a9d-4865-9a56-cedecd0ed40e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:02.124889 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39134d-1a9d-4865-9a56-cedecd0ed40e service swift] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:02.125262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39134d-1a9d-4865-9a56-cedecd0ed40e service swift] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:02.125542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39134d-1a9d-4865-9a56-cedecd0ed40e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:02.183866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e675b-2ef2-4aea-b27b-f9b316e4f617 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:02.196953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19489d61-e11b-4fe9-ac62-121bf59964cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:02.197686 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19489d61-e11b-4fe9-ac62-121bf59964cf service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:02.197976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19489d61-e11b-4fe9-ac62-121bf59964cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:02.198257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19489d61-e11b-4fe9-ac62-121bf59964cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:02.225426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19489d61-e11b-4fe9-ac62-121bf59964cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:02.233002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19489d61-e11b-4fe9-ac62-121bf59964cf service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:02.271334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 884/1767] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:02.288430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19489d61-e11b-4fe9-ac62-121bf59964cf service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:02.291408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-19489d61-e11b-4fe9-ac62-121bf59964cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:02.319138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39134d-1a9d-4865-9a56-cedecd0ed40e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:02.325188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39134d-1a9d-4865-9a56-cedecd0ed40e service swift] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:02.351575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19489d61-e11b-4fe9-ac62-121bf59964cf service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:02.364536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 884/1768] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:20:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:20:02.373831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:02.378640 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:02.378936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:02.379223 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] PATH_INFO: `/v3/users/67b63ee1c8144b66b001ba5ce1398fc5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:02.396786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67b63ee1c8144b66b001ba5ce1398fc5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:02.401496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:02.551392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67b63ee1c8144b66b001ba5ce1398fc5'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:02.574610 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67b63ee1c8144b66b001ba5ce1398fc5'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:02.580963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67b63ee1c8144b66b001ba5ce1398fc5'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:02.608172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67b63ee1c8144b66b001ba5ce1398fc5'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:02.612504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67b63ee1c8144b66b001ba5ce1398fc5'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:02.638101 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 885/1769] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:02 2022] DELETE /identity/v3/users/67b63ee1c8144b66b001ba5ce1398fc5 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:02.642603 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd7b41-64c6-4a05-9ff1-0348a67657ce None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:02.643101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd7b41-64c6-4a05-9ff1-0348a67657ce None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:02.643567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd7b41-64c6-4a05-9ff1-0348a67657ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:02.703209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-a7cd7b41-64c6-4a05-9ff1-0348a67657ce None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:02.709146 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a7cd7b41-64c6-4a05-9ff1-0348a67657ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:02.935728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 885/1770] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:02.959809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:02.962429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:03.155011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44125-84f3-432f-a775-cba0efb68b27 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:03.172574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedbbb2-8e85-4d95-bfaa-665ca535a9ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:03.173259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedbbb2-8e85-4d95-bfaa-665ca535a9ec service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:03.173550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedbbb2-8e85-4d95-bfaa-665ca535a9ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:03.173831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedbbb2-8e85-4d95-bfaa-665ca535a9ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:03.228270 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baedbbb2-8e85-4d95-bfaa-665ca535a9ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:03.235790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baedbbb2-8e85-4d95-bfaa-665ca535a9ec service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:03.278801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 886/1771] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:03.684359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd7b41-64c6-4a05-9ff1-0348a67657ce None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:03.686340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a7cd7b41-64c6-4a05-9ff1-0348a67657ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:03.710093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd7b41-64c6-4a05-9ff1-0348a67657ce None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:03.736847 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce852097-6157-4504-98cf-bc866e4f142e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:03.737817 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce852097-6157-4504-98cf-bc866e4f142e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:03.738020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce852097-6157-4504-98cf-bc866e4f142e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:03.738515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce852097-6157-4504-98cf-bc866e4f142e admin admin] PATH_INFO: `/v3/projects/362b426c8f7d4bf7a292aed5d1c0eaf6` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:03.744365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce852097-6157-4504-98cf-bc866e4f142e admin admin] RBAC: Authorizing `identity:delete_project(project_id=362b426c8f7d4bf7a292aed5d1c0eaf6)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:03.747765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce852097-6157-4504-98cf-bc866e4f142e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:03.954964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ce852097-6157-4504-98cf-bc866e4f142e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '362b426c8f7d4bf7a292aed5d1c0eaf6', 'request_id': 'req-ce852097-6157-4504-98cf-bc866e4f142e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:03.969139 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ce852097-6157-4504-98cf-bc866e4f142e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '362b426c8f7d4bf7a292aed5d1c0eaf6', 'request_id': 'req-ce852097-6157-4504-98cf-bc866e4f142e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:03.996920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ce852097-6157-4504-98cf-bc866e4f142e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '362b426c8f7d4bf7a292aed5d1c0eaf6', 'request_id': 'req-ce852097-6157-4504-98cf-bc866e4f142e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:04.020674 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:04.035784 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ce852097-6157-4504-98cf-bc866e4f142e admin admin] The token cache is being invalidate because project 362b426c8f7d4bf7a292aed5d1c0eaf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:04.036926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ce852097-6157-4504-98cf-bc866e4f142e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:04.038894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ce852097-6157-4504-98cf-bc866e4f142e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:04.043846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 886/1772] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:03 2022] DELETE /identity/v3/projects/362b426c8f7d4bf7a292aed5d1c0eaf6 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:04.047941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedbbb2-8e85-4d95-bfaa-665ca535a9ec service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:04.079504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedbbb2-8e85-4d95-bfaa-665ca535a9ec service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:04.229809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:04.388860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:04.393900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:04.396489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:04.400375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] PATH_INFO: `/v3/users/024f7a86883348a09eab7de3570f0b8a` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:04.409481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] RBAC: Authorizing `identity:delete_user(user_id=024f7a86883348a09eab7de3570f0b8a)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:04.417884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:04.607106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024f7a86883348a09eab7de3570f0b8a'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:04.627079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e1614-29aa-42d5-a89d-95e2ca34bd94 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:04.629335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e1614-29aa-42d5-a89d-95e2ca34bd94 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:04.629335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e1614-29aa-42d5-a89d-95e2ca34bd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:04.647830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024f7a86883348a09eab7de3570f0b8a'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:04.648834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024f7a86883348a09eab7de3570f0b8a'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:04.678113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ed4e1614-29aa-42d5-a89d-95e2ca34bd94 None None] MFA Rules not processed for user `7e0797679c07454abc867cd7b30c8b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:04.705316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ed4e1614-29aa-42d5-a89d-95e2ca34bd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:04.724634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024f7a86883348a09eab7de3570f0b8a'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:04.730560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024f7a86883348a09eab7de3570f0b8a'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:04.801063 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 887/1773] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:04 2022] DELETE /identity/v3/users/024f7a86883348a09eab7de3570f0b8a => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:04.879362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 887/1774] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Jun 17 07:20:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:04.988906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:04.990822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:05.297064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a121254-df23-496b-a57d-c04a7538338f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:05.309858 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:05.311319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:05.312630 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:05.312940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] PATH_INFO: `/v3/projects/05e314048d714af09aca36b6c1b15cc2` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:05.319907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05e314048d714af09aca36b6c1b15cc2)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:05.321933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:05.372256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e1614-29aa-42d5-a89d-95e2ca34bd94 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:05.374173 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ed4e1614-29aa-42d5-a89d-95e2ca34bd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:05.396614 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e1614-29aa-42d5-a89d-95e2ca34bd94 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:05.502552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05e314048d714af09aca36b6c1b15cc2', 'request_id': 'req-8ab76467-4292-46d0-a692-06efa02b1678'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:05.509165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:05.529600 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05e314048d714af09aca36b6c1b15cc2', 'request_id': 'req-8ab76467-4292-46d0-a692-06efa02b1678'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:05.549233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05e314048d714af09aca36b6c1b15cc2', 'request_id': 'req-8ab76467-4292-46d0-a692-06efa02b1678'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:05.600148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] The token cache is being invalidate because project 05e314048d714af09aca36b6c1b15cc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:05.600448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:05.608812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:05.611275 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 888/1775] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:04 2022] DELETE /identity/v3/projects/05e314048d714af09aca36b6c1b15cc2 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:05.617332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:05.625806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:05.697117 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff2e51-e181-46af-8ced-8924aa111527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:05.699717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff2e51-e181-46af-8ced-8924aa111527 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:05.700612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff2e51-e181-46af-8ced-8924aa111527 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:05.703317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff2e51-e181-46af-8ced-8924aa111527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:05.709562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9eff2e51-e181-46af-8ced-8924aa111527 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:05.778472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff2e51-e181-46af-8ced-8924aa111527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:05.788878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff2e51-e181-46af-8ced-8924aa111527 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:05.841610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 888/1776] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:20:05.844010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff2e51-e181-46af-8ced-8924aa111527 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:05.853271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9eff2e51-e181-46af-8ced-8924aa111527 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:05.923554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff2e51-e181-46af-8ced-8924aa111527 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:06.034638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab76467-4292-46d0-a692-06efa02b1678 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:06.061601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53923e31-ceb0-4f6b-ae9b-9b6adef92484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:06.062570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53923e31-ceb0-4f6b-ae9b-9b6adef92484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:06.076097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53923e31-ceb0-4f6b-ae9b-9b6adef92484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:06.076444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53923e31-ceb0-4f6b-ae9b-9b6adef92484 admin admin] PATH_INFO: `/v3/users/af4d8a80ff3e43ba9aba1b7d926d3ac2` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:06.079005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53923e31-ceb0-4f6b-ae9b-9b6adef92484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af4d8a80ff3e43ba9aba1b7d926d3ac2)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:06.085717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53923e31-ceb0-4f6b-ae9b-9b6adef92484 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:06.208506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:06.210477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:06.210774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:06.212698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] PATH_INFO: `/v3/users/ed56fb506b4144059c28f5802a73f28c` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:06.214782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed56fb506b4144059c28f5802a73f28c)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:06.221717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:06.268803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-53923e31-ceb0-4f6b-ae9b-9b6adef92484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af4d8a80ff3e43ba9aba1b7d926d3ac2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:06.304629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-53923e31-ceb0-4f6b-ae9b-9b6adef92484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af4d8a80ff3e43ba9aba1b7d926d3ac2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:06.305820 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-53923e31-ceb0-4f6b-ae9b-9b6adef92484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af4d8a80ff3e43ba9aba1b7d926d3ac2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:06.358359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-53923e31-ceb0-4f6b-ae9b-9b6adef92484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af4d8a80ff3e43ba9aba1b7d926d3ac2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:06.361367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-53923e31-ceb0-4f6b-ae9b-9b6adef92484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af4d8a80ff3e43ba9aba1b7d926d3ac2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:06.412574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed56fb506b4144059c28f5802a73f28c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:06.414818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed56fb506b4144059c28f5802a73f28c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:06.418302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:DELETE=8,keystone:SELECT=92 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:06.441279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed56fb506b4144059c28f5802a73f28c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:06.467269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 889/1777] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:05 2022] DELETE /identity/v3/users/af4d8a80ff3e43ba9aba1b7d926d3ac2 => generated 0 bytes in 847 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:06.470799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed56fb506b4144059c28f5802a73f28c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:06.472438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed56fb506b4144059c28f5802a73f28c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:06.538002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 889/1778] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:05 2022] DELETE /identity/v3/users/ed56fb506b4144059c28f5802a73f28c => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:06.542530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c8eb8-a8f1-4302-86ba-3de0b058ee0f None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:06.542821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c8eb8-a8f1-4302-86ba-3de0b058ee0f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:06.543105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c8eb8-a8f1-4302-86ba-3de0b058ee0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:06.622746 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-791c8eb8-a8f1-4302-86ba-3de0b058ee0f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:06.624012 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-791c8eb8-a8f1-4302-86ba-3de0b058ee0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:06.863081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:06.891877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 890/1779] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:07.003006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:07.007157 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:07.181252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ada290-7e99-4299-88c2-3d92ad971955 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:07.193057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78e7d8-801a-49c0-a052-06e95879d4df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:07.193993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78e7d8-801a-49c0-a052-06e95879d4df service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:07.194509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78e7d8-801a-49c0-a052-06e95879d4df service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:07.194899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78e7d8-801a-49c0-a052-06e95879d4df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:07.217605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df78e7d8-801a-49c0-a052-06e95879d4df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:07.219309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df78e7d8-801a-49c0-a052-06e95879d4df service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:07.268161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 890/1780] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:07.665821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c8eb8-a8f1-4302-86ba-3de0b058ee0f None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:07.669200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-791c8eb8-a8f1-4302-86ba-3de0b058ee0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:07.700837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c8eb8-a8f1-4302-86ba-3de0b058ee0f None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:07.709897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:07.711430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:07.723092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:07.724145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] PATH_INFO: `/v3/projects/3e5e2e90d8454a19a2db67bcf4eeac59` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:07.725555 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e5e2e90d8454a19a2db67bcf4eeac59)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:07.728139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:07.968742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e5e2e90d8454a19a2db67bcf4eeac59', 'request_id': 'req-7303fc00-b201-4621-9b00-613134b9cf13'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:07.992630 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e5e2e90d8454a19a2db67bcf4eeac59', 'request_id': 'req-7303fc00-b201-4621-9b00-613134b9cf13'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:08.003816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e5e2e90d8454a19a2db67bcf4eeac59', 'request_id': 'req-7303fc00-b201-4621-9b00-613134b9cf13'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:08.026158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] The token cache is being invalidate because project 3e5e2e90d8454a19a2db67bcf4eeac59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:08.028413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:08.030285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:08.037690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78e7d8-801a-49c0-a052-06e95879d4df service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:08.039669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 891/1781] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:07 2022] DELETE /identity/v3/projects/3e5e2e90d8454a19a2db67bcf4eeac59 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:08.059228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78e7d8-801a-49c0-a052-06e95879d4df service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:08.232144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:08.234301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:08.236468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:08.239378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] PATH_INFO: `/v3/users/54e9add447ae404e95387bffd87b2abb` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:08.247785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=54e9add447ae404e95387bffd87b2abb)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:08.255568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:08.435862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54e9add447ae404e95387bffd87b2abb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:08.437479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54e9add447ae404e95387bffd87b2abb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:08.454816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54e9add447ae404e95387bffd87b2abb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:08.487167 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:08.499531 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54e9add447ae404e95387bffd87b2abb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:08.499892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54e9add447ae404e95387bffd87b2abb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:08.520050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 891/1782] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:08 2022] DELETE /identity/v3/users/54e9add447ae404e95387bffd87b2abb => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:08.788991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:08.791170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:08.809885 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303fc00-b201-4621-9b00-613134b9cf13 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:08.955282 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e716a-daff-4589-94cd-c2acad2e768b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:08.958109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e716a-daff-4589-94cd-c2acad2e768b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:08.958837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e716a-daff-4589-94cd-c2acad2e768b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:08.961082 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e716a-daff-4589-94cd-c2acad2e768b admin admin] PATH_INFO: `/v3/projects/452c022a71a64a50b9123bfee135cb56` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:08.963730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e716a-daff-4589-94cd-c2acad2e768b admin admin] RBAC: Authorizing `identity:delete_project(project_id=452c022a71a64a50b9123bfee135cb56)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:08.967929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e716a-daff-4589-94cd-c2acad2e768b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:09.113791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-830e716a-daff-4589-94cd-c2acad2e768b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '452c022a71a64a50b9123bfee135cb56', 'request_id': 'req-830e716a-daff-4589-94cd-c2acad2e768b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.129259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-830e716a-daff-4589-94cd-c2acad2e768b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '452c022a71a64a50b9123bfee135cb56', 'request_id': 'req-830e716a-daff-4589-94cd-c2acad2e768b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.149520 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-830e716a-daff-4589-94cd-c2acad2e768b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '452c022a71a64a50b9123bfee135cb56', 'request_id': 'req-830e716a-daff-4589-94cd-c2acad2e768b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.164176 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:09.165727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:09.174203 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-830e716a-daff-4589-94cd-c2acad2e768b admin admin] The token cache is being invalidate because project 452c022a71a64a50b9123bfee135cb56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:09.174530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-830e716a-daff-4589-94cd-c2acad2e768b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.175182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-830e716a-daff-4589-94cd-c2acad2e768b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.177271 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 892/1783] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:08 2022] DELETE /identity/v3/projects/452c022a71a64a50b9123bfee135cb56 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:09.182947 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b65ac-c05a-4586-b787-eac99f3e6f09 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:09.183137 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b65ac-c05a-4586-b787-eac99f3e6f09 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:09.183405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b65ac-c05a-4586-b787-eac99f3e6f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:09.191959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1375c59-a7ee-4c4d-a895-4fc74ded0bda admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:09.213520 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-e45b65ac-c05a-4586-b787-eac99f3e6f09 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:09.214917 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e45b65ac-c05a-4586-b787-eac99f3e6f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:09.229222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:09.372865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 893/1784] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:09.387952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331585db-6145-44ed-a697-6ff4b2915984 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:09.387952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331585db-6145-44ed-a697-6ff4b2915984 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:09.387952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331585db-6145-44ed-a697-6ff4b2915984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:09.420513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-331585db-6145-44ed-a697-6ff4b2915984 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:09.424344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-331585db-6145-44ed-a697-6ff4b2915984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:09.429720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:09.431587 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:09.431986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:09.432499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a admin admin] PATH_INFO: `/v3/projects/6c67b87260f4469c9d244dafa43b8857` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:09.433816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c67b87260f4469c9d244dafa43b8857)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:09.435988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:09.445809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 894/1785] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:09.450763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331585db-6145-44ed-a697-6ff4b2915984 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:09.469813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331585db-6145-44ed-a697-6ff4b2915984 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:09.485671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015831aa-11a1-4173-8e9a-7a7a47113262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:09.486460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015831aa-11a1-4173-8e9a-7a7a47113262 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:09.486823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015831aa-11a1-4173-8e9a-7a7a47113262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:09.487628 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015831aa-11a1-4173-8e9a-7a7a47113262 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:09.488685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015831aa-11a1-4173-8e9a-7a7a47113262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:09.489169 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015831aa-11a1-4173-8e9a-7a7a47113262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:09.491892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015831aa-11a1-4173-8e9a-7a7a47113262 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:09.516749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 895/1786] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:20:09.519097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1923e-7fa1-40ad-be72-2cfe1cff5df6 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:09.519466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1923e-7fa1-40ad-be72-2cfe1cff5df6 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:09.519812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1923e-7fa1-40ad-be72-2cfe1cff5df6 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:09.522446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 896/1787] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:20:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:20:09.524296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c67b87260f4469c9d244dafa43b8857', 'request_id': 'req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.526339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1923e-7fa1-40ad-be72-2cfe1cff5df6 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:09.544361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c67b87260f4469c9d244dafa43b8857', 'request_id': 'req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.557479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c67b87260f4469c9d244dafa43b8857', 'request_id': 'req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.572302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a admin admin] The token cache is being invalidate because project 6c67b87260f4469c9d244dafa43b8857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:09.573460 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.574579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1a9df665-8edb-4030-a3ec-2dc8f1f2094a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.582125 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 892/1788] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:09 2022] DELETE /identity/v3/projects/6c67b87260f4469c9d244dafa43b8857 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:09.583835 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9831af5-7647-4abc-be79-a762455f61cc None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:09.584229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9831af5-7647-4abc-be79-a762455f61cc None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:09.584582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9831af5-7647-4abc-be79-a762455f61cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:09.626312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-a9831af5-7647-4abc-be79-a762455f61cc None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:09.627829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a9831af5-7647-4abc-be79-a762455f61cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:09.675892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1923e-7fa1-40ad-be72-2cfe1cff5df6 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:09.690385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fcd94-fe09-4a16-93ac-e230ed4ffea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:09.691189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fcd94-fe09-4a16-93ac-e230ed4ffea5 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:09.695594 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fcd94-fe09-4a16-93ac-e230ed4ffea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:09.696006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fcd94-fe09-4a16-93ac-e230ed4ffea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:09.697530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fcd94-fe09-4a16-93ac-e230ed4ffea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:09.699688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fcd94-fe09-4a16-93ac-e230ed4ffea5 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:09.721711 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 893/1789] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:20:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:09.725164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9831af5-7647-4abc-be79-a762455f61cc None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:09.727261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a9831af5-7647-4abc-be79-a762455f61cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:09.745016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9831af5-7647-4abc-be79-a762455f61cc None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:09.749334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 897/1790] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:09 2022] POST /identity/v3/projects => generated 363 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:09.751672 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:09.753510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fcd94-fe09-4a16-93ac-e230ed4ffea5 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:09.753776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:09.754252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:09.754725 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] PATH_INFO: `/v3/users/4b9d9d836cf04984b92842d150c1d382` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:09.755297 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cc3fcd94-fe09-4a16-93ac-e230ed4ffea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:09.758184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b9d9d836cf04984b92842d150c1d382)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:09.762891 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:09.766613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:09.887816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fcd94-fe09-4a16-93ac-e230ed4ffea5 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:09.899243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ff176-58bb-4857-8ad9-d86043e8376f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:09.899887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ff176-58bb-4857-8ad9-d86043e8376f service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:09.900206 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ff176-58bb-4857-8ad9-d86043e8376f service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:09.900493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ff176-58bb-4857-8ad9-d86043e8376f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:09.902519 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b9d9d836cf04984b92842d150c1d382'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.903271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b9d9d836cf04984b92842d150c1d382'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.912237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b9d9d836cf04984b92842d150c1d382'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.915713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ff176-58bb-4857-8ad9-d86043e8376f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:09.920504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ff176-58bb-4857-8ad9-d86043e8376f service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:09.926152 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b9d9d836cf04984b92842d150c1d382'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.927173 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b9d9d836cf04984b92842d150c1d382'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:09.932818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 898/1791] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:09.942743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 894/1792] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:09 2022] DELETE /identity/v3/users/4b9d9d836cf04984b92842d150c1d382 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:09.945915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:09.947117 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ff176-58bb-4857-8ad9-d86043e8376f service nova] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:09.948664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:09.949434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-594ff176-58bb-4857-8ad9-d86043e8376f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:09.958084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:10.079644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb922a-280b-40fd-a4d0-c66d07155abc admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:10.089414 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42dc276-7982-4bfa-9733-2a505582e17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:10.090349 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42dc276-7982-4bfa-9733-2a505582e17b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:10.090637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42dc276-7982-4bfa-9733-2a505582e17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:10.090915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42dc276-7982-4bfa-9733-2a505582e17b admin admin] PATH_INFO: `/v3/projects/a9496f21d58140b59861774da9e944d6` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:10.092290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42dc276-7982-4bfa-9733-2a505582e17b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9496f21d58140b59861774da9e944d6)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:10.094486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42dc276-7982-4bfa-9733-2a505582e17b admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:10.099279 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ff176-58bb-4857-8ad9-d86043e8376f service nova] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:10.115171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3429c30-794b-4ad7-9030-75781d1df5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:10.115889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3429c30-794b-4ad7-9030-75781d1df5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:10.116254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3429c30-794b-4ad7-9030-75781d1df5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:10.116542 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3429c30-794b-4ad7-9030-75781d1df5c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:10.117343 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3429c30-794b-4ad7-9030-75781d1df5c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:10.119998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3429c30-794b-4ad7-9030-75781d1df5c5 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:10.159494 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 899/1793] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:09 2022] POST /identity/v3/users => generated 282 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:10.161412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3429c30-794b-4ad7-9030-75781d1df5c5 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:10.163568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b3429c30-794b-4ad7-9030-75781d1df5c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:10.178147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3429c30-794b-4ad7-9030-75781d1df5c5 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:10.187771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b42dc276-7982-4bfa-9733-2a505582e17b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9496f21d58140b59861774da9e944d6', 'request_id': 'req-b42dc276-7982-4bfa-9733-2a505582e17b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:10.198957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b42dc276-7982-4bfa-9733-2a505582e17b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9496f21d58140b59861774da9e944d6', 'request_id': 'req-b42dc276-7982-4bfa-9733-2a505582e17b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:10.207251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b42dc276-7982-4bfa-9733-2a505582e17b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9496f21d58140b59861774da9e944d6', 'request_id': 'req-b42dc276-7982-4bfa-9733-2a505582e17b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:10.219757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b42dc276-7982-4bfa-9733-2a505582e17b admin admin] The token cache is being invalidate because project a9496f21d58140b59861774da9e944d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:10.220050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b42dc276-7982-4bfa-9733-2a505582e17b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:10.221112 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b42dc276-7982-4bfa-9733-2a505582e17b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:10.226798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 895/1794] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:09 2022] DELETE /identity/v3/projects/a9496f21d58140b59861774da9e944d6 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:10.231119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548e0e5-2c5d-4782-a3fa-f7a8a6c6c3d1 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:10.231119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548e0e5-2c5d-4782-a3fa-f7a8a6c6c3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:10.231119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548e0e5-2c5d-4782-a3fa-f7a8a6c6c3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:10.256728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f548e0e5-2c5d-4782-a3fa-f7a8a6c6c3d1 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:10.258834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f548e0e5-2c5d-4782-a3fa-f7a8a6c6c3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:10.274911 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:10.281793 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c277936-f2db-4739-bf80-002bb6522af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:10.283130 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c277936-f2db-4739-bf80-002bb6522af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:10.283378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c277936-f2db-4739-bf80-002bb6522af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:10.283652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c277936-f2db-4739-bf80-002bb6522af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:10.284624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c277936-f2db-4739-bf80-002bb6522af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:10.288872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c277936-f2db-4739-bf80-002bb6522af0 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:10.309661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 900/1795] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:10.313352 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c277936-f2db-4739-bf80-002bb6522af0 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:10.316844 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3c277936-f2db-4739-bf80-002bb6522af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:10.338048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c277936-f2db-4739-bf80-002bb6522af0 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:10.354448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33beb35-98aa-430a-83cc-58ebcfcf8dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:10.355289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33beb35-98aa-430a-83cc-58ebcfcf8dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:10.355431 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33beb35-98aa-430a-83cc-58ebcfcf8dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:10.355869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33beb35-98aa-430a-83cc-58ebcfcf8dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:10.356668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33beb35-98aa-430a-83cc-58ebcfcf8dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:10.358332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33beb35-98aa-430a-83cc-58ebcfcf8dcf admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:10.385499 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 901/1796] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:10.387292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33beb35-98aa-430a-83cc-58ebcfcf8dcf admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:10.398355 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33beb35-98aa-430a-83cc-58ebcfcf8dcf admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:10.405790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71887c9-e901-49fa-bf47-3544a5e58124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:10.406926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71887c9-e901-49fa-bf47-3544a5e58124 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:10.407403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71887c9-e901-49fa-bf47-3544a5e58124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:10.407851 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71887c9-e901-49fa-bf47-3544a5e58124 admin admin] PATH_INFO: `/v3/projects/f0aeef30bcff4955bfe810c64681039e/users/2300fc8980714c7ca051b702522aee48/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:10.409746 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 896/1797] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:10.418615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8954c-8c82-472c-b2db-e11bc7518910 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:10.419002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8954c-8c82-472c-b2db-e11bc7518910 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:10.419292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8954c-8c82-472c-b2db-e11bc7518910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:10.437121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71887c9-e901-49fa-bf47-3544a5e58124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0aeef30bcff4955bfe810c64681039e, user_id=2300fc8980714c7ca051b702522aee48, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:10.441319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71887c9-e901-49fa-bf47-3544a5e58124 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:10.449683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-66b8954c-8c82-472c-b2db-e11bc7518910 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:10.450972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-66b8954c-8c82-472c-b2db-e11bc7518910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:10.475593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 902/1798] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:10 2022] PUT /identity/v3/projects/f0aeef30bcff4955bfe810c64681039e/users/2300fc8980714c7ca051b702522aee48/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:10.491861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e64ee8-84f6-44f3-b0ba-1fd699e6d569 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:10.492292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e64ee8-84f6-44f3-b0ba-1fd699e6d569 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:10.492643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e64ee8-84f6-44f3-b0ba-1fd699e6d569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:10.527586 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-e9e64ee8-84f6-44f3-b0ba-1fd699e6d569 None None] MFA Rules not processed for user `2300fc8980714c7ca051b702522aee48`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:10.528864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e9e64ee8-84f6-44f3-b0ba-1fd699e6d569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:10.572773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 897/1799] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:20:10.573868 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8954c-8c82-472c-b2db-e11bc7518910 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:10.575200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-66b8954c-8c82-472c-b2db-e11bc7518910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:10.580905 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:10.593986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8954c-8c82-472c-b2db-e11bc7518910 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:10.599627 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:10.600573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:10.600870 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:10.601933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] PATH_INFO: `/v3/projects/b0e7a45ffa5449539fbe11539064ffb0` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:10.603863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0e7a45ffa5449539fbe11539064ffb0)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:10.606059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:10.617734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 903/1800] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:10 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:20:10.618949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e64ee8-84f6-44f3-b0ba-1fd699e6d569 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:10.636613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e64ee8-84f6-44f3-b0ba-1fd699e6d569 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:10.642440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf63690-16f9-4c45-bc89-b87c7914d88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:10.643250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf63690-16f9-4c45-bc89-b87c7914d88d admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:10.643615 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf63690-16f9-4c45-bc89-b87c7914d88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:10.643957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf63690-16f9-4c45-bc89-b87c7914d88d admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:10.645278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf63690-16f9-4c45-bc89-b87c7914d88d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:10.645754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf63690-16f9-4c45-bc89-b87c7914d88d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:10.648249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf63690-16f9-4c45-bc89-b87c7914d88d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:10.666025 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 904/1801] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:20:10.667453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf63690-16f9-4c45-bc89-b87c7914d88d admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:10.681075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf63690-16f9-4c45-bc89-b87c7914d88d admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:10.681605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0e7a45ffa5449539fbe11539064ffb0', 'request_id': 'req-55537646-35cd-46aa-b468-9e0dab3353bb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:10.696228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0e7a45ffa5449539fbe11539064ffb0', 'request_id': 'req-55537646-35cd-46aa-b468-9e0dab3353bb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:10.704170 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0e7a45ffa5449539fbe11539064ffb0', 'request_id': 'req-55537646-35cd-46aa-b468-9e0dab3353bb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:10.729217 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] The token cache is being invalidate because project b0e7a45ffa5449539fbe11539064ffb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:10.730537 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:10.731260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:10.734013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 898/1802] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:10 2022] DELETE /identity/v3/projects/b0e7a45ffa5449539fbe11539064ffb0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:10.745464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:10.747984 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:10.787160 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281112f-ac12-49ea-8154-bd54b380cbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:10.787944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281112f-ac12-49ea-8154-bd54b380cbc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:10.790942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281112f-ac12-49ea-8154-bd54b380cbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:10.791235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281112f-ac12-49ea-8154-bd54b380cbc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:10.795788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281112f-ac12-49ea-8154-bd54b380cbc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:10.798567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281112f-ac12-49ea-8154-bd54b380cbc8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:10.838053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:10.864093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 905/1803] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:10 2022] POST /identity/v3/projects => generated 363 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:10.864885 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de9594-6241-47f3-a9bb-3f68039bcf8c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:10.865181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de9594-6241-47f3-a9bb-3f68039bcf8c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:10.865464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de9594-6241-47f3-a9bb-3f68039bcf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:10.900342 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55537646-35cd-46aa-b468-9e0dab3353bb admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:10.908778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-17de9594-6241-47f3-a9bb-3f68039bcf8c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:10.926365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-17de9594-6241-47f3-a9bb-3f68039bcf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:11.047353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 906/1804] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:11.047353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de9594-6241-47f3-a9bb-3f68039bcf8c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.047353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-17de9594-6241-47f3-a9bb-3f68039bcf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:11.055947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc84df5-2ede-42cd-b54f-c85362843e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.056470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de9594-6241-47f3-a9bb-3f68039bcf8c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:11.056667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc84df5-2ede-42cd-b54f-c85362843e8e admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.056957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc84df5-2ede-42cd-b54f-c85362843e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.057377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc84df5-2ede-42cd-b54f-c85362843e8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.058174 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc84df5-2ede-42cd-b54f-c85362843e8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.060788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc84df5-2ede-42cd-b54f-c85362843e8e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.067024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fe70e-ac78-42c4-a6a5-7095db546f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.067872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fe70e-ac78-42c4-a6a5-7095db546f10 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.068206 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fe70e-ac78-42c4-a6a5-7095db546f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.068540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fe70e-ac78-42c4-a6a5-7095db546f10 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.069258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fe70e-ac78-42c4-a6a5-7095db546f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.072644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fe70e-ac78-42c4-a6a5-7095db546f10 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.102823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 907/1805] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:11 2022] POST /identity/v3/users => generated 288 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:11.103999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930cf8c-98de-4ee6-94e4-923ed823c690 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.104364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930cf8c-98de-4ee6-94e4-923ed823c690 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.104900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930cf8c-98de-4ee6-94e4-923ed823c690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.120340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 899/1806] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:10 2022] POST /identity/v3/projects => generated 367 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:11.120340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc84df5-2ede-42cd-b54f-c85362843e8e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.135574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-b930cf8c-98de-4ee6-94e4-923ed823c690 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:11.137163 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b930cf8c-98de-4ee6-94e4-923ed823c690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:11.204952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:11.217334 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc84df5-2ede-42cd-b54f-c85362843e8e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:11.223128 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958da605-a894-4fef-b903-6656b05d2cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.223913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958da605-a894-4fef-b903-6656b05d2cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.224450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958da605-a894-4fef-b903-6656b05d2cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.224802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958da605-a894-4fef-b903-6656b05d2cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.225617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958da605-a894-4fef-b903-6656b05d2cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.227351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958da605-a894-4fef-b903-6656b05d2cc9 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.248468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 900/1807] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:11.249824 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958da605-a894-4fef-b903-6656b05d2cc9 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.262121 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958da605-a894-4fef-b903-6656b05d2cc9 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:11.267376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc565e51-f695-4a3e-b422-ade46c3ce026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.268099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc565e51-f695-4a3e-b422-ade46c3ce026 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.268547 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc565e51-f695-4a3e-b422-ade46c3ce026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.268771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc565e51-f695-4a3e-b422-ade46c3ce026 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.269841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc565e51-f695-4a3e-b422-ade46c3ce026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.272833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc565e51-f695-4a3e-b422-ade46c3ce026 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.297723 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 908/1808] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:11.297723 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930cf8c-98de-4ee6-94e4-923ed823c690 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.308510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930cf8c-98de-4ee6-94e4-923ed823c690 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:11.312008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 901/1809] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:11 2022] POST /identity/v3/users => generated 284 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:11.313239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc565e51-f695-4a3e-b422-ade46c3ce026 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.316979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77b673-515e-4d53-a548-5d361dfa3144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.317901 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77b673-515e-4d53-a548-5d361dfa3144 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.318189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77b673-515e-4d53-a548-5d361dfa3144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.318476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77b673-515e-4d53-a548-5d361dfa3144 admin admin] PATH_INFO: `/v3/projects/d5b2867f246648508d42ca0bb14b00f2/users/67309fa73b0d4cf8b659ce7a0b6c6c35/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.326524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc565e51-f695-4a3e-b422-ade46c3ce026 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:11.332390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7df7e4-3a82-4d09-84df-2bb1c106d6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.333498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7df7e4-3a82-4d09-84df-2bb1c106d6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.334057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7df7e4-3a82-4d09-84df-2bb1c106d6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.334411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7df7e4-3a82-4d09-84df-2bb1c106d6c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.335227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7df7e4-3a82-4d09-84df-2bb1c106d6c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:11.335612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7df7e4-3a82-4d09-84df-2bb1c106d6c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.336053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77b673-515e-4d53-a548-5d361dfa3144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b2867f246648508d42ca0bb14b00f2, user_id=67309fa73b0d4cf8b659ce7a0b6c6c35, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.337497 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7df7e4-3a82-4d09-84df-2bb1c106d6c2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.339338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77b673-515e-4d53-a548-5d361dfa3144 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.369667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 909/1810] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:11 2022] PUT /identity/v3/projects/d5b2867f246648508d42ca0bb14b00f2/users/67309fa73b0d4cf8b659ce7a0b6c6c35/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:11.373333 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77b673-515e-4d53-a548-5d361dfa3144 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.387708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 902/1811] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:20:11.389641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7df7e4-3a82-4d09-84df-2bb1c106d6c2 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.415375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7df7e4-3a82-4d09-84df-2bb1c106d6c2 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:11.464696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:11.505641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77b673-515e-4d53-a548-5d361dfa3144 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:11.515748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56227e-d68c-46c2-834f-93e47c9bbe61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.516775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56227e-d68c-46c2-834f-93e47c9bbe61 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.517134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56227e-d68c-46c2-834f-93e47c9bbe61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.517488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56227e-d68c-46c2-834f-93e47c9bbe61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.518336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56227e-d68c-46c2-834f-93e47c9bbe61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.519056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf7566-6474-4e10-8de4-0e339e917c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.520359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56227e-d68c-46c2-834f-93e47c9bbe61 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.524531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf7566-6474-4e10-8de4-0e339e917c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.525211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf7566-6474-4e10-8de4-0e339e917c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.528743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf7566-6474-4e10-8de4-0e339e917c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.530781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf7566-6474-4e10-8de4-0e339e917c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.533429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf7566-6474-4e10-8de4-0e339e917c4e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.544163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 903/1812] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:11.547656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 910/1813] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:11.548796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf7566-6474-4e10-8de4-0e339e917c4e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.548929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56227e-d68c-46c2-834f-93e47c9bbe61 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.561019 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf7566-6474-4e10-8de4-0e339e917c4e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:11.566340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc8dfe-aabb-4d35-83a2-631c802c6e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.567641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc8dfe-aabb-4d35-83a2-631c802c6e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.568111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc8dfe-aabb-4d35-83a2-631c802c6e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.569214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc8dfe-aabb-4d35-83a2-631c802c6e3e admin admin] PATH_INFO: `/v3/projects/d5b2867f246648508d42ca0bb14b00f2/users/67309fa73b0d4cf8b659ce7a0b6c6c35/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.572804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc8dfe-aabb-4d35-83a2-631c802c6e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b2867f246648508d42ca0bb14b00f2, user_id=67309fa73b0d4cf8b659ce7a0b6c6c35, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.575697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc8dfe-aabb-4d35-83a2-631c802c6e3e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.579364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56227e-d68c-46c2-834f-93e47c9bbe61 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:11.591670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da753e3-3c6e-46e3-9109-68b162dd4ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.592380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da753e3-3c6e-46e3-9109-68b162dd4ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.592671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da753e3-3c6e-46e3-9109-68b162dd4ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.592953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da753e3-3c6e-46e3-9109-68b162dd4ae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.594231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da753e3-3c6e-46e3-9109-68b162dd4ae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.599745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da753e3-3c6e-46e3-9109-68b162dd4ae6 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.602288 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 911/1814] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:11 2022] PUT /identity/v3/projects/d5b2867f246648508d42ca0bb14b00f2/users/67309fa73b0d4cf8b659ce7a0b6c6c35/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:11.603578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc8dfe-aabb-4d35-83a2-631c802c6e3e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.660586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 904/1815] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:11 2022] POST /identity/v3/projects => generated 409 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:11.663070 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e42d83-14b1-4320-8c77-3efcda01a243 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.663912 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e42d83-14b1-4320-8c77-3efcda01a243 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.664527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e42d83-14b1-4320-8c77-3efcda01a243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.697418 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-89e42d83-14b1-4320-8c77-3efcda01a243 None None] MFA Rules not processed for user `67309fa73b0d4cf8b659ce7a0b6c6c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:11.699090 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-89e42d83-14b1-4320-8c77-3efcda01a243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:11.700620 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc8dfe-aabb-4d35-83a2-631c802c6e3e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:11.755296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:11.775731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c379dba-2703-4320-9b7b-0f0c6a180266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.776585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c379dba-2703-4320-9b7b-0f0c6a180266 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.776931 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c379dba-2703-4320-9b7b-0f0c6a180266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.777356 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c379dba-2703-4320-9b7b-0f0c6a180266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.778168 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c379dba-2703-4320-9b7b-0f0c6a180266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.782514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c379dba-2703-4320-9b7b-0f0c6a180266 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.807016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 912/1816] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:11.807952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c379dba-2703-4320-9b7b-0f0c6a180266 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.818899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c379dba-2703-4320-9b7b-0f0c6a180266 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:11.828228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b25841-d26b-409b-97cd-1732182f55bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.828996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b25841-d26b-409b-97cd-1732182f55bb admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.829286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b25841-d26b-409b-97cd-1732182f55bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.829568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b25841-d26b-409b-97cd-1732182f55bb admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.830411 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b25841-d26b-409b-97cd-1732182f55bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.832420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b25841-d26b-409b-97cd-1732182f55bb admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.845928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 905/1817] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:11 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:11.847632 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e42d83-14b1-4320-8c77-3efcda01a243 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.876007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 913/1818] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:11 2022] POST /identity/v3/users => generated 305 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:11.877625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b25841-d26b-409b-97cd-1732182f55bb admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:11.880904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-a4b25841-d26b-409b-97cd-1732182f55bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:11.884476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e42d83-14b1-4320-8c77-3efcda01a243 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:11.889523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35df536-62a3-48ad-b0c5-efe47ad8f07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:11.890843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35df536-62a3-48ad-b0c5-efe47ad8f07a admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:11.890963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35df536-62a3-48ad-b0c5-efe47ad8f07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:11.891382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35df536-62a3-48ad-b0c5-efe47ad8f07a admin admin] PATH_INFO: `/v3/projects/7fcb281c60c442ff9181f1993959d8c3/users/ce9f0e19d76a440c8b7365fc1449d41b/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:11.929897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35df536-62a3-48ad-b0c5-efe47ad8f07a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fcb281c60c442ff9181f1993959d8c3, user_id=ce9f0e19d76a440c8b7365fc1449d41b, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:11.934872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35df536-62a3-48ad-b0c5-efe47ad8f07a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:11.957519 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 906/1819] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:11 2022] PUT /identity/v3/projects/7fcb281c60c442ff9181f1993959d8c3/users/ce9f0e19d76a440c8b7365fc1449d41b/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:11.964645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35df536-62a3-48ad-b0c5-efe47ad8f07a admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:12.007926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b25841-d26b-409b-97cd-1732182f55bb admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:12.114834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ae27e-47ef-4eaf-af32-072b7cd506b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:12.115552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ae27e-47ef-4eaf-af32-072b7cd506b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:12.115841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ae27e-47ef-4eaf-af32-072b7cd506b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:12.116281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ae27e-47ef-4eaf-af32-072b7cd506b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:12.126884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35df536-62a3-48ad-b0c5-efe47ad8f07a admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:12.137508 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47cf54-d61c-45b2-9491-469d719a3922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:12.138691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47cf54-d61c-45b2-9491-469d719a3922 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:12.139804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47cf54-d61c-45b2-9491-469d719a3922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:12.140240 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47cf54-d61c-45b2-9491-469d719a3922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:12.141362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47cf54-d61c-45b2-9491-469d719a3922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:12.144494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47cf54-d61c-45b2-9491-469d719a3922 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:12.166079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 907/1820] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:12.178663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36727837-7b0e-4a2a-ace7-92043b6d31d9 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:12.178988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36727837-7b0e-4a2a-ace7-92043b6d31d9 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:12.179283 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36727837-7b0e-4a2a-ace7-92043b6d31d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:12.207151 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:12.268045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-36727837-7b0e-4a2a-ace7-92043b6d31d9 None None] MFA Rules not processed for user `ce9f0e19d76a440c8b7365fc1449d41b`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:12.269268 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ae27e-47ef-4eaf-af32-072b7cd506b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:12.271470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ae27e-47ef-4eaf-af32-072b7cd506b0 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:12.271849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-36727837-7b0e-4a2a-ace7-92043b6d31d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:12.298014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 914/1821] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:11 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:12.298916 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ae27e-47ef-4eaf-af32-072b7cd506b0 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:12.330571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ae27e-47ef-4eaf-af32-072b7cd506b0 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:12.343685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e956ce-f684-440b-a2ff-82798c172877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:12.344859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e956ce-f684-440b-a2ff-82798c172877 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:12.345377 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e956ce-f684-440b-a2ff-82798c172877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:12.345829 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e956ce-f684-440b-a2ff-82798c172877 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:12.346638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e956ce-f684-440b-a2ff-82798c172877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:12.348378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e956ce-f684-440b-a2ff-82798c172877 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:12.379677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 915/1822] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:12.396790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e956ce-f684-440b-a2ff-82798c172877 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:12.421785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e956ce-f684-440b-a2ff-82798c172877 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:12.445131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67226c02-2859-49f8-bfee-ea369e85cea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:12.446999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67226c02-2859-49f8-bfee-ea369e85cea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:12.446999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67226c02-2859-49f8-bfee-ea369e85cea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:12.447186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67226c02-2859-49f8-bfee-ea369e85cea8 admin admin] PATH_INFO: `/v3/projects/a7a3e4f73f204fb99efd5e62842beb75/users/41120c85efc94fe7ab8e14469949df2f/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:12.478886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 908/1823] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:12.483647 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36727837-7b0e-4a2a-ace7-92043b6d31d9 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:12.485164 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67226c02-2859-49f8-bfee-ea369e85cea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a3e4f73f204fb99efd5e62842beb75, user_id=41120c85efc94fe7ab8e14469949df2f, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:12.488189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67226c02-2859-49f8-bfee-ea369e85cea8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:12.492148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:12.497895 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36727837-7b0e-4a2a-ace7-92043b6d31d9 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:12.504292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee48e08-3c5a-4dd3-8409-a0a2b2d553b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:12.505014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48e08-3c5a-4dd3-8409-a0a2b2d553b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:12.505306 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48e08-3c5a-4dd3-8409-a0a2b2d553b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:12.505586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48e08-3c5a-4dd3-8409-a0a2b2d553b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:12.507043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee48e08-3c5a-4dd3-8409-a0a2b2d553b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:12.509185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee48e08-3c5a-4dd3-8409-a0a2b2d553b7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:12.529148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 916/1824] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:12 2022] PUT /identity/v3/projects/a7a3e4f73f204fb99efd5e62842beb75/users/41120c85efc94fe7ab8e14469949df2f/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:12.558407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cbfa5-68d2-44c7-9bf4-65f51c092833 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:12.558718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cbfa5-68d2-44c7-9bf4-65f51c092833 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:12.558998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cbfa5-68d2-44c7-9bf4-65f51c092833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:12.575713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 909/1825] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:12 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:12.577710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee48e08-3c5a-4dd3-8409-a0a2b2d553b7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:12.592528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-867cbfa5-68d2-44c7-9bf4-65f51c092833 None None] MFA Rules not processed for user `41120c85efc94fe7ab8e14469949df2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:12.593923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-867cbfa5-68d2-44c7-9bf4-65f51c092833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:12.718554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee48e08-3c5a-4dd3-8409-a0a2b2d553b7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:12.721983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 917/1826] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:12 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:12.725886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cbfa5-68d2-44c7-9bf4-65f51c092833 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:12.743001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c25d0-2c38-4472-82de-2d7f912721b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:12.751496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c25d0-2c38-4472-82de-2d7f912721b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:12.751860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c25d0-2c38-4472-82de-2d7f912721b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:12.763173 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c25d0-2c38-4472-82de-2d7f912721b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:12.764048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c25d0-2c38-4472-82de-2d7f912721b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:12.766035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c25d0-2c38-4472-82de-2d7f912721b6 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:12.770292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cbfa5-68d2-44c7-9bf4-65f51c092833 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:12.799273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cda18f-ff2c-4970-adf1-dd30b417acc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:12.799273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda18f-ff2c-4970-adf1-dd30b417acc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:12.799273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda18f-ff2c-4970-adf1-dd30b417acc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:12.799273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda18f-ff2c-4970-adf1-dd30b417acc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:12.799983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cda18f-ff2c-4970-adf1-dd30b417acc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:12.811170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cda18f-ff2c-4970-adf1-dd30b417acc0 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:12.834982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 910/1827] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:12 2022] POST /identity/v3/users => generated 289 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:12.839178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c25d0-2c38-4472-82de-2d7f912721b6 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:12.881397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c25d0-2c38-4472-82de-2d7f912721b6 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:12.893019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273413c4-57a6-4a91-b38e-6a9e3f0c78ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:12.893691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 918/1828] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:12 2022] POST /identity/v3/projects => generated 409 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:20:12.893811 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273413c4-57a6-4a91-b38e-6a9e3f0c78ae admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:12.893992 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273413c4-57a6-4a91-b38e-6a9e3f0c78ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:12.894274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273413c4-57a6-4a91-b38e-6a9e3f0c78ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:12.895063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273413c4-57a6-4a91-b38e-6a9e3f0c78ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:12.895220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cda18f-ff2c-4970-adf1-dd30b417acc0 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:12.898774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273413c4-57a6-4a91-b38e-6a9e3f0c78ae admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:12.925126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 911/1829] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:12.930031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273413c4-57a6-4a91-b38e-6a9e3f0c78ae admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:12.965247 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273413c4-57a6-4a91-b38e-6a9e3f0c78ae admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:13.065078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cda18f-ff2c-4970-adf1-dd30b417acc0 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:13.076173 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:13.083143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa5bd8-7578-4ab6-b956-6f99b2ebff19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:13.083845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa5bd8-7578-4ab6-b956-6f99b2ebff19 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:13.084157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa5bd8-7578-4ab6-b956-6f99b2ebff19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:13.084469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa5bd8-7578-4ab6-b956-6f99b2ebff19 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:13.085207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa5bd8-7578-4ab6-b956-6f99b2ebff19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:13.097411 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa5bd8-7578-4ab6-b956-6f99b2ebff19 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:13.100382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2428f4-1c0c-44a9-8f6d-f1f7ae08a48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:13.101749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2428f4-1c0c-44a9-8f6d-f1f7ae08a48b admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:13.102339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2428f4-1c0c-44a9-8f6d-f1f7ae08a48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:13.102669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2428f4-1c0c-44a9-8f6d-f1f7ae08a48b admin admin] PATH_INFO: `/v3/projects/9e7e103180e34d0fa958d0c1b3cd62ef/users/cb1c6bfff18c48a5897483e0572213dc/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:13.128783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2428f4-1c0c-44a9-8f6d-f1f7ae08a48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7e103180e34d0fa958d0c1b3cd62ef, user_id=cb1c6bfff18c48a5897483e0572213dc, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:13.131864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2428f4-1c0c-44a9-8f6d-f1f7ae08a48b admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:13.180017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 919/1830] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:12 2022] POST /identity/v3/users => generated 311 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:13.181767 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa5bd8-7578-4ab6-b956-6f99b2ebff19 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:13.199808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa5bd8-7578-4ab6-b956-6f99b2ebff19 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:13.203565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 912/1831] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:12 2022] PUT /identity/v3/projects/9e7e103180e34d0fa958d0c1b3cd62ef/users/cb1c6bfff18c48a5897483e0572213dc/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:13.206941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2428f4-1c0c-44a9-8f6d-f1f7ae08a48b admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:13.442903 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:13.490292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:13.494780 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:13.495131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:13.495427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] PATH_INFO: `/v3/users/2300fc8980714c7ca051b702522aee48` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:13.503869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2300fc8980714c7ca051b702522aee48)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:13.509443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2428f4-1c0c-44a9-8f6d-f1f7ae08a48b admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:13.511789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:13.526102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9638765-3c2a-47ca-9978-fd45fb61081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:13.529869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9638765-3c2a-47ca-9978-fd45fb61081f admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:13.530961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9638765-3c2a-47ca-9978-fd45fb61081f admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:13.531307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9638765-3c2a-47ca-9978-fd45fb61081f admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:13.532095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9638765-3c2a-47ca-9978-fd45fb61081f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:13.533988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9638765-3c2a-47ca-9978-fd45fb61081f admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:13.569757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 913/1832] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:13.577262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9638765-3c2a-47ca-9978-fd45fb61081f admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:13.603886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9638765-3c2a-47ca-9978-fd45fb61081f admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:13.617299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338b96b-7e47-43c6-8f05-ba3dffbdbee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:13.618318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338b96b-7e47-43c6-8f05-ba3dffbdbee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:13.619730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338b96b-7e47-43c6-8f05-ba3dffbdbee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:13.620515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338b96b-7e47-43c6-8f05-ba3dffbdbee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:13.624599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338b96b-7e47-43c6-8f05-ba3dffbdbee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:13.628877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338b96b-7e47-43c6-8f05-ba3dffbdbee6 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:13.658129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 914/1833] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:13.658129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338b96b-7e47-43c6-8f05-ba3dffbdbee6 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:13.682421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338b96b-7e47-43c6-8f05-ba3dffbdbee6 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:13.696483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a1094-0971-4b9c-85da-65019af513ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:13.697410 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a1094-0971-4b9c-85da-65019af513ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:13.697695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a1094-0971-4b9c-85da-65019af513ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:13.698005 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a1094-0971-4b9c-85da-65019af513ee admin admin] PATH_INFO: `/v3/projects/bfb6cee2478d4091b8ca22c9f47c5ab9/users/cd5d778038c04cc4ac516ee87637d781/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:13.701131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2300fc8980714c7ca051b702522aee48'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:13.720955 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2300fc8980714c7ca051b702522aee48'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:13.727069 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2300fc8980714c7ca051b702522aee48'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:13.748338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a1094-0971-4b9c-85da-65019af513ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb6cee2478d4091b8ca22c9f47c5ab9, user_id=cd5d778038c04cc4ac516ee87637d781, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:13.751400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a1094-0971-4b9c-85da-65019af513ee admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:13.764431 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2300fc8980714c7ca051b702522aee48'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:13.765170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2300fc8980714c7ca051b702522aee48'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:13.787255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 915/1834] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:13 2022] PUT /identity/v3/projects/bfb6cee2478d4091b8ca22c9f47c5ab9/users/cd5d778038c04cc4ac516ee87637d781/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:13.787255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a1094-0971-4b9c-85da-65019af513ee admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:13.788769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-1e5a1094-0971-4b9c-85da-65019af513ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:13.798554 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 920/1835] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:13 2022] DELETE /identity/v3/users/2300fc8980714c7ca051b702522aee48 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:13.801391 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:13.807502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:13.959909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a1094-0971-4b9c-85da-65019af513ee admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:14.104846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270aa72d-51c1-4118-ad0b-840d2676d6f8 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:14.120219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2bc5d-48fb-4a34-8189-fe688bdd5394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:14.122714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2bc5d-48fb-4a34-8189-fe688bdd5394 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:14.123922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2bc5d-48fb-4a34-8189-fe688bdd5394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:14.124240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2bc5d-48fb-4a34-8189-fe688bdd5394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:14.125973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2bc5d-48fb-4a34-8189-fe688bdd5394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:14.133182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2bc5d-48fb-4a34-8189-fe688bdd5394 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:14.149723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:14.177449 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 921/1836] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:14.179495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badc706-bfc1-4708-aae3-ed55608f2aa2 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:14.179794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badc706-bfc1-4708-aae3-ed55608f2aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:14.181014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badc706-bfc1-4708-aae3-ed55608f2aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:14.209418 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f991be-ca19-4a8f-bc57-150871c12b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:14.210349 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f991be-ca19-4a8f-bc57-150871c12b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:14.210626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f991be-ca19-4a8f-bc57-150871c12b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:14.210904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f991be-ca19-4a8f-bc57-150871c12b95 admin admin] PATH_INFO: `/v3/projects/9e7e103180e34d0fa958d0c1b3cd62ef/users/cb1c6bfff18c48a5897483e0572213dc/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:14.215725 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f991be-ca19-4a8f-bc57-150871c12b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7e103180e34d0fa958d0c1b3cd62ef, user_id=cb1c6bfff18c48a5897483e0572213dc, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:14.218716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f991be-ca19-4a8f-bc57-150871c12b95 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:14.259804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 916/1837] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:13 2022] PUT /identity/v3/projects/9e7e103180e34d0fa958d0c1b3cd62ef/users/cb1c6bfff18c48a5897483e0572213dc/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:14.260879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f991be-ca19-4a8f-bc57-150871c12b95 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:14.361010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6badc706-bfc1-4708-aae3-ed55608f2aa2 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:14.367965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6badc706-bfc1-4708-aae3-ed55608f2aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:14.434863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f991be-ca19-4a8f-bc57-150871c12b95 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:14.452995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5437dcb-5ea0-4fc8-b5fc-23469c7e5c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:14.453851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5437dcb-5ea0-4fc8-b5fc-23469c7e5c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:14.454146 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5437dcb-5ea0-4fc8-b5fc-23469c7e5c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:14.454429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5437dcb-5ea0-4fc8-b5fc-23469c7e5c53 admin admin] PATH_INFO: `/v3/projects/bfb6cee2478d4091b8ca22c9f47c5ab9/users/cd5d778038c04cc4ac516ee87637d781/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:14.459274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5437dcb-5ea0-4fc8-b5fc-23469c7e5c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb6cee2478d4091b8ca22c9f47c5ab9, user_id=cd5d778038c04cc4ac516ee87637d781, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:14.465964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5437dcb-5ea0-4fc8-b5fc-23469c7e5c53 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:14.500997 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 917/1838] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:14 2022] PUT /identity/v3/projects/bfb6cee2478d4091b8ca22c9f47c5ab9/users/cd5d778038c04cc4ac516ee87637d781/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:14.507601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48ea9c-7a55-4511-b6a1-23ec9656cf1e None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:14.507929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48ea9c-7a55-4511-b6a1-23ec9656cf1e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:14.508253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48ea9c-7a55-4511-b6a1-23ec9656cf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:14.589359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-bc48ea9c-7a55-4511-b6a1-23ec9656cf1e None None] MFA Rules not processed for user `cb1c6bfff18c48a5897483e0572213dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:14.590888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-bc48ea9c-7a55-4511-b6a1-23ec9656cf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:14.606457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 922/1839] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:14.607965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af24b7-5871-487d-b08e-7e5e87642861 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:14.608285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af24b7-5871-487d-b08e-7e5e87642861 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:14.608610 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af24b7-5871-487d-b08e-7e5e87642861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:14.635145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:14.664743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-08af24b7-5871-487d-b08e-7e5e87642861 None None] MFA Rules not processed for user `cd5d778038c04cc4ac516ee87637d781`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:14.675999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-08af24b7-5871-487d-b08e-7e5e87642861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:14.788140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 918/1840] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:14.791335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48ea9c-7a55-4511-b6a1-23ec9656cf1e None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:14.795261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-bc48ea9c-7a55-4511-b6a1-23ec9656cf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:14.828870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 923/1841] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:14 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:14.830124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af24b7-5871-487d-b08e-7e5e87642861 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:14.831621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-08af24b7-5871-487d-b08e-7e5e87642861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:14.938976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af24b7-5871-487d-b08e-7e5e87642861 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:14.944402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52228c5b-23ba-4cd1-9d60-53f2efc112d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:14.945467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52228c5b-23ba-4cd1-9d60-53f2efc112d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:14.945858 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52228c5b-23ba-4cd1-9d60-53f2efc112d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:14.947089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52228c5b-23ba-4cd1-9d60-53f2efc112d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:14.959820 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52228c5b-23ba-4cd1-9d60-53f2efc112d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:14.962283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52228c5b-23ba-4cd1-9d60-53f2efc112d0 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:14.993399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 924/1842] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:14.995528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52228c5b-23ba-4cd1-9d60-53f2efc112d0 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:14.998796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-52228c5b-23ba-4cd1-9d60-53f2efc112d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:15.014451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52228c5b-23ba-4cd1-9d60-53f2efc112d0 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:15.019100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197db38-caea-46d1-91f9-e5db0b0d53f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:15.019837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197db38-caea-46d1-91f9-e5db0b0d53f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:15.020480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197db38-caea-46d1-91f9-e5db0b0d53f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:15.020818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197db38-caea-46d1-91f9-e5db0b0d53f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:15.065990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197db38-caea-46d1-91f9-e5db0b0d53f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:15.070164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48ea9c-7a55-4511-b6a1-23ec9656cf1e None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:15.074218 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197db38-caea-46d1-91f9-e5db0b0d53f7 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:15.088942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9603d2-1a90-4847-95bc-d5a124667aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:15.093224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9603d2-1a90-4847-95bc-d5a124667aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:15.093559 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9603d2-1a90-4847-95bc-d5a124667aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:15.116585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9603d2-1a90-4847-95bc-d5a124667aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:15.148508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 925/1843] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:14 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:15.154345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:15.216829 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197db38-caea-46d1-91f9-e5db0b0d53f7 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:15.245327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197db38-caea-46d1-91f9-e5db0b0d53f7 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:15.264243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275fd4d-d4c0-45d1-8c35-c2566a2e0955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:15.266217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275fd4d-d4c0-45d1-8c35-c2566a2e0955 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:15.268191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275fd4d-d4c0-45d1-8c35-c2566a2e0955 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:15.271434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275fd4d-d4c0-45d1-8c35-c2566a2e0955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:15.278457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1275fd4d-d4c0-45d1-8c35-c2566a2e0955 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:15.358107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9603d2-1a90-4847-95bc-d5a124667aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:15.361794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9603d2-1a90-4847-95bc-d5a124667aa0 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:15.444497 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 919/1844] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:15.480193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275fd4d-d4c0-45d1-8c35-c2566a2e0955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:15.481669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275fd4d-d4c0-45d1-8c35-c2566a2e0955 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:15.516104 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 926/1845] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:20:15.733016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9603d2-1a90-4847-95bc-d5a124667aa0 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:15.747921 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9603d2-1a90-4847-95bc-d5a124667aa0 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:15.759275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5eab85-c1ec-4fbb-ab72-20e9d511c195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:15.762201 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5eab85-c1ec-4fbb-ab72-20e9d511c195 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:15.762553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5eab85-c1ec-4fbb-ab72-20e9d511c195 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:15.762840 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5eab85-c1ec-4fbb-ab72-20e9d511c195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:15.765981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-5f5eab85-c1ec-4fbb-ab72-20e9d511c195 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:15.941483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5eab85-c1ec-4fbb-ab72-20e9d511c195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:15.943358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5eab85-c1ec-4fbb-ab72-20e9d511c195 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:15.966731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 920/1846] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:17.699490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275fd4d-d4c0-45d1-8c35-c2566a2e0955 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:17.701884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1275fd4d-d4c0-45d1-8c35-c2566a2e0955 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:17.728940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275fd4d-d4c0-45d1-8c35-c2566a2e0955 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:17.742549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783071a-92ab-42f3-a59d-44c2c091068c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:17.743742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783071a-92ab-42f3-a59d-44c2c091068c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:17.744037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783071a-92ab-42f3-a59d-44c2c091068c admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:17.744409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783071a-92ab-42f3-a59d-44c2c091068c admin admin] PATH_INFO: `/v3/users/4336f95f0b0c483e92db5a8ac058c9c1` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:17.748300 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6783071a-92ab-42f3-a59d-44c2c091068c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4336f95f0b0c483e92db5a8ac058c9c1)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:17.750524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6783071a-92ab-42f3-a59d-44c2c091068c admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:17.776381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5eab85-c1ec-4fbb-ab72-20e9d511c195 service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:17.777885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-5f5eab85-c1ec-4fbb-ab72-20e9d511c195 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:17.801912 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:17.805111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5eab85-c1ec-4fbb-ab72-20e9d511c195 service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:17.831469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:17.832465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:17.833337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:17.833337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] PATH_INFO: `/v3/projects/f0aeef30bcff4955bfe810c64681039e` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:17.835087 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0aeef30bcff4955bfe810c64681039e)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:17.862276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:17.873850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6783071a-92ab-42f3-a59d-44c2c091068c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4336f95f0b0c483e92db5a8ac058c9c1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:17.891428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6783071a-92ab-42f3-a59d-44c2c091068c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4336f95f0b0c483e92db5a8ac058c9c1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:17.892713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6783071a-92ab-42f3-a59d-44c2c091068c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4336f95f0b0c483e92db5a8ac058c9c1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:17.924772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6783071a-92ab-42f3-a59d-44c2c091068c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4336f95f0b0c483e92db5a8ac058c9c1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:17.926793 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6783071a-92ab-42f3-a59d-44c2c091068c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4336f95f0b0c483e92db5a8ac058c9c1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:17.957856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 927/1847] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:17 2022] DELETE /identity/v3/users/4336f95f0b0c483e92db5a8ac058c9c1 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:17.962979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203839fd-88e6-4cc3-99b1-034f0c2c2ea8 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:17.963277 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203839fd-88e6-4cc3-99b1-034f0c2c2ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:17.963557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203839fd-88e6-4cc3-99b1-034f0c2c2ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:17.980588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0aeef30bcff4955bfe810c64681039e', 'request_id': 'req-fc30640f-aa95-43d2-a0fc-a8e34efad99c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:18.018009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0aeef30bcff4955bfe810c64681039e', 'request_id': 'req-fc30640f-aa95-43d2-a0fc-a8e34efad99c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:18.022706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-203839fd-88e6-4cc3-99b1-034f0c2c2ea8 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:18.023834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-203839fd-88e6-4cc3-99b1-034f0c2c2ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:18.046270 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0aeef30bcff4955bfe810c64681039e', 'request_id': 'req-fc30640f-aa95-43d2-a0fc-a8e34efad99c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:18.084789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] The token cache is being invalidate because project f0aeef30bcff4955bfe810c64681039e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:18.085946 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:18.096463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:18.103608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 921/1848] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:17 2022] DELETE /identity/v3/projects/f0aeef30bcff4955bfe810c64681039e => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:18.105598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:18.108216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:18.177337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:18.194200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 928/1849] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:18.195681 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c3f4d-cbb2-40d2-8c0a-1b8f5d204650 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:18.196569 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c3f4d-cbb2-40d2-8c0a-1b8f5d204650 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:18.196862 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c3f4d-cbb2-40d2-8c0a-1b8f5d204650 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:18.207279 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 929/1850] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:20:18 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:20:18.234443 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c3f4d-cbb2-40d2-8c0a-1b8f5d204650 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:18.237700 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e30c3f4d-cbb2-40d2-8c0a-1b8f5d204650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:18.276379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30640f-aa95-43d2-a0fc-a8e34efad99c admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:18.290858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:18.293045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:18.300293 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:18.300577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] PATH_INFO: `/v3/users/67309fa73b0d4cf8b659ce7a0b6c6c35` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:18.304950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67309fa73b0d4cf8b659ce7a0b6c6c35)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:18.319924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:18.425633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c3f4d-cbb2-40d2-8c0a-1b8f5d204650 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:18.433589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb1e4e-bb1c-4a3c-b534-7942e98885a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:18.434378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb1e4e-bb1c-4a3c-b534-7942e98885a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:18.434924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb1e4e-bb1c-4a3c-b534-7942e98885a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:18.435544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb1e4e-bb1c-4a3c-b534-7942e98885a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:18.461233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb1e4e-bb1c-4a3c-b534-7942e98885a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:18.465057 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb1e4e-bb1c-4a3c-b534-7942e98885a7 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:18.474514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67309fa73b0d4cf8b659ce7a0b6c6c35'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:18.475358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67309fa73b0d4cf8b659ce7a0b6c6c35'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:18.501222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 930/1851] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:18.503020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d86fa-d260-4f8d-8869-fc0a40fb4b11 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:18.503416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d86fa-d260-4f8d-8869-fc0a40fb4b11 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:18.503852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d86fa-d260-4f8d-8869-fc0a40fb4b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:18.505099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67309fa73b0d4cf8b659ce7a0b6c6c35'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:18.529982 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-563d86fa-d260-4f8d-8869-fc0a40fb4b11 None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:18.532223 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-563d86fa-d260-4f8d-8869-fc0a40fb4b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:18.567496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67309fa73b0d4cf8b659ce7a0b6c6c35'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:18.568896 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67309fa73b0d4cf8b659ce7a0b6c6c35'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:18.593347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:18.606212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 922/1852] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:18 2022] DELETE /identity/v3/users/67309fa73b0d4cf8b659ce7a0b6c6c35 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:18.642803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 931/1853] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:20:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:19.102832 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:19.107078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:19.247793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d085ee9-3441-4a6f-810b-86692ebf6cd2 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:19.254836 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d693b-3029-48e8-b704-db311f2898fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:19.255592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d693b-3029-48e8-b704-db311f2898fb service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:19.255958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d693b-3029-48e8-b704-db311f2898fb service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:19.256580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d693b-3029-48e8-b704-db311f2898fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:19.292177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d693b-3029-48e8-b704-db311f2898fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:19.294764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d693b-3029-48e8-b704-db311f2898fb service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:19.335414 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 923/1854] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:19.885798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d86fa-d260-4f8d-8869-fc0a40fb4b11 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:19.889177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-563d86fa-d260-4f8d-8869-fc0a40fb4b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:19.921356 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d86fa-d260-4f8d-8869-fc0a40fb4b11 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:19.926490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d693b-3029-48e8-b704-db311f2898fb service nova] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:19.928426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-293d693b-3029-48e8-b704-db311f2898fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:20.066680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:20.113617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2313695-9165-4cdb-a16a-15826dc02a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:20.114598 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2313695-9165-4cdb-a16a-15826dc02a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:20.114929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2313695-9165-4cdb-a16a-15826dc02a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:20.115237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2313695-9165-4cdb-a16a-15826dc02a82 admin admin] PATH_INFO: `/v3/projects/d5b2867f246648508d42ca0bb14b00f2` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:20.119271 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2313695-9165-4cdb-a16a-15826dc02a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5b2867f246648508d42ca0bb14b00f2)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:20.121531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2313695-9165-4cdb-a16a-15826dc02a82 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:20.125268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d693b-3029-48e8-b704-db311f2898fb service nova] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:20.132698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:20.133886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:20.134546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:20.135094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] PATH_INFO: `/v3/projects/5ab9e078b5da4de285e3b82cf3d79e08` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:20.137097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab9e078b5da4de285e3b82cf3d79e08)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:20.139706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:20.217868 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab9e078b5da4de285e3b82cf3d79e08', 'request_id': 'req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.228720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab9e078b5da4de285e3b82cf3d79e08', 'request_id': 'req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.237222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab9e078b5da4de285e3b82cf3d79e08', 'request_id': 'req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.240723 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-f2313695-9165-4cdb-a16a-15826dc02a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5b2867f246648508d42ca0bb14b00f2', 'request_id': 'req-f2313695-9165-4cdb-a16a-15826dc02a82'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.265264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-f2313695-9165-4cdb-a16a-15826dc02a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5b2867f246648508d42ca0bb14b00f2', 'request_id': 'req-f2313695-9165-4cdb-a16a-15826dc02a82'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.267005 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] The token cache is being invalidate because project 5ab9e078b5da4de285e3b82cf3d79e08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:20.269826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.272272 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.279334 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 924/1855] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:19 2022] DELETE /identity/v3/projects/5ab9e078b5da4de285e3b82cf3d79e08 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:20.283591 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:20.287850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:20.290413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-f2313695-9165-4cdb-a16a-15826dc02a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5b2867f246648508d42ca0bb14b00f2', 'request_id': 'req-f2313695-9165-4cdb-a16a-15826dc02a82'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.338022 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-f2313695-9165-4cdb-a16a-15826dc02a82 admin admin] The token cache is being invalidate because project d5b2867f246648508d42ca0bb14b00f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:20.338316 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-f2313695-9165-4cdb-a16a-15826dc02a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.344676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-f2313695-9165-4cdb-a16a-15826dc02a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.349852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 932/1856] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:19 2022] DELETE /identity/v3/projects/d5b2867f246648508d42ca0bb14b00f2 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:20.357188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ee917-dc03-4475-bcc5-5765593f9eab None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:20.357486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ee917-dc03-4475-bcc5-5765593f9eab None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:20.357773 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ee917-dc03-4475-bcc5-5765593f9eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:20.410586 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-7d2ee917-dc03-4475-bcc5-5765593f9eab None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:20.412273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7d2ee917-dc03-4475-bcc5-5765593f9eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:20.429036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8957d7b-3a4a-415d-8f88-8c80d5c2657e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:20.537812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:20.556418 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 933/1857] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:20.564892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb230be-f1fb-4e96-b790-93acd3cc9bdf None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:20.565178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb230be-f1fb-4e96-b790-93acd3cc9bdf None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:20.565461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb230be-f1fb-4e96-b790-93acd3cc9bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:20.585412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-dbb230be-f1fb-4e96-b790-93acd3cc9bdf None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:20.587141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-dbb230be-f1fb-4e96-b790-93acd3cc9bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:20.593236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:20.594352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:20.594704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:20.595044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] PATH_INFO: `/v3/users/b5c29d7121a44c3f85abc61d1eda8320` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:20.596350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5c29d7121a44c3f85abc61d1eda8320)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:20.598828 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:20.599041 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 934/1858] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:20:20.600157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb230be-f1fb-4e96-b790-93acd3cc9bdf None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:20.615609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb230be-f1fb-4e96-b790-93acd3cc9bdf None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:20.621352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:20.626197 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8986d6a-0555-457f-9dfe-3934c5b52495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:20.628275 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8986d6a-0555-457f-9dfe-3934c5b52495 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:20.629770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8986d6a-0555-457f-9dfe-3934c5b52495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:20.630042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8986d6a-0555-457f-9dfe-3934c5b52495 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:20.630860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8986d6a-0555-457f-9dfe-3934c5b52495 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:20.631191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8986d6a-0555-457f-9dfe-3934c5b52495 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:20.633471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8986d6a-0555-457f-9dfe-3934c5b52495 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:20.658604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 935/1859] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:20:20.663192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8986d6a-0555-457f-9dfe-3934c5b52495 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:20.674756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5c29d7121a44c3f85abc61d1eda8320'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.675431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5c29d7121a44c3f85abc61d1eda8320'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.682211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8986d6a-0555-457f-9dfe-3934c5b52495 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:20.685420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5c29d7121a44c3f85abc61d1eda8320'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.690966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac42c7-37b1-4c4b-aa51-0ff921442420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:20.691984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac42c7-37b1-4c4b-aa51-0ff921442420 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:20.692458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac42c7-37b1-4c4b-aa51-0ff921442420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:20.693021 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac42c7-37b1-4c4b-aa51-0ff921442420 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:20.693879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac42c7-37b1-4c4b-aa51-0ff921442420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:20.696208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac42c7-37b1-4c4b-aa51-0ff921442420 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:20.702017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5c29d7121a44c3f85abc61d1eda8320'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.706125 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5c29d7121a44c3f85abc61d1eda8320'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:20.727631 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 925/1860] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:20 2022] DELETE /identity/v3/users/b5c29d7121a44c3f85abc61d1eda8320 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:20.749256 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 936/1861] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:20 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:20.753557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:20.754658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:20.847473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fb35f-e471-41d8-bee4-0058eda4421e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:20.923166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373d499-4e78-4c59-8a22-eb3aac191a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:20.923930 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373d499-4e78-4c59-8a22-eb3aac191a5c admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:20.924325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373d499-4e78-4c59-8a22-eb3aac191a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:20.924692 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373d499-4e78-4c59-8a22-eb3aac191a5c admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:20.925484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373d499-4e78-4c59-8a22-eb3aac191a5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:20.927579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373d499-4e78-4c59-8a22-eb3aac191a5c admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:20.959097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 926/1862] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:20 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:20:20.960801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac42c7-37b1-4c4b-aa51-0ff921442420 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:20.972396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac42c7-37b1-4c4b-aa51-0ff921442420 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:20.978877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599915c6-8615-4306-9a5a-d3669fb9d182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:20.979782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599915c6-8615-4306-9a5a-d3669fb9d182 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:20.980155 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599915c6-8615-4306-9a5a-d3669fb9d182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:20.980540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599915c6-8615-4306-9a5a-d3669fb9d182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:20.981460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599915c6-8615-4306-9a5a-d3669fb9d182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:20.983069 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599915c6-8615-4306-9a5a-d3669fb9d182 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:20.994185 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 937/1863] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:20.998133 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373d499-4e78-4c59-8a22-eb3aac191a5c admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:21.008586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373d499-4e78-4c59-8a22-eb3aac191a5c admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:21.014663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f4286-b57a-4b0b-bead-73b3580530fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:21.015391 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f4286-b57a-4b0b-bead-73b3580530fe admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.016260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f4286-b57a-4b0b-bead-73b3580530fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.016536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f4286-b57a-4b0b-bead-73b3580530fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.017284 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f4286-b57a-4b0b-bead-73b3580530fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:21.019061 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f4286-b57a-4b0b-bead-73b3580530fe admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:21.037069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 927/1864] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:21.041372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599915c6-8615-4306-9a5a-d3669fb9d182 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:21.051947 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599915c6-8615-4306-9a5a-d3669fb9d182 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:21.061208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af421db-d51e-4160-8c34-7850d39e09aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:21.061619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f4286-b57a-4b0b-bead-73b3580530fe admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:21.062529 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af421db-d51e-4160-8c34-7850d39e09aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.062917 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af421db-d51e-4160-8c34-7850d39e09aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.063356 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af421db-d51e-4160-8c34-7850d39e09aa admin admin] PATH_INFO: `/v3/projects/8ac76d0685db4a5cac0605892884deaf/users/1575eb17ca494619bf67c7c8b4ec94a7/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.063473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-424f4286-b57a-4b0b-bead-73b3580530fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:21.078850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f4286-b57a-4b0b-bead-73b3580530fe admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:21.083124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af421db-d51e-4160-8c34-7850d39e09aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac76d0685db4a5cac0605892884deaf, user_id=1575eb17ca494619bf67c7c8b4ec94a7, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:21.084490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2f116-07af-4358-b44d-e7d1429dd345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:21.085499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2f116-07af-4358-b44d-e7d1429dd345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.085836 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2f116-07af-4358-b44d-e7d1429dd345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.086054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2f116-07af-4358-b44d-e7d1429dd345 admin admin] PATH_INFO: `/v3/projects/0de8e4142bed424d8845afcd9dbeaf01` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.087792 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2f116-07af-4358-b44d-e7d1429dd345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de8e4142bed424d8845afcd9dbeaf01)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:21.087919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af421db-d51e-4160-8c34-7850d39e09aa admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:21.089721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2f116-07af-4358-b44d-e7d1429dd345 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:21.103775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:21.116493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 938/1865] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:21 2022] PUT /identity/v3/projects/8ac76d0685db4a5cac0605892884deaf/users/1575eb17ca494619bf67c7c8b4ec94a7/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:21.127048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2a80d-a9c6-48bf-8744-9e2151de578a None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.127375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2a80d-a9c6-48bf-8744-9e2151de578a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.127613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2a80d-a9c6-48bf-8744-9e2151de578a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.154140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-42e2f116-07af-4358-b44d-e7d1429dd345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de8e4142bed424d8845afcd9dbeaf01', 'request_id': 'req-42e2f116-07af-4358-b44d-e7d1429dd345'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:21.160810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-42e2f116-07af-4358-b44d-e7d1429dd345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de8e4142bed424d8845afcd9dbeaf01', 'request_id': 'req-42e2f116-07af-4358-b44d-e7d1429dd345'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:21.163664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-21d2a80d-a9c6-48bf-8744-9e2151de578a None None] MFA Rules not processed for user `1575eb17ca494619bf67c7c8b4ec94a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:21.165491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-21d2a80d-a9c6-48bf-8744-9e2151de578a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:21.169756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-42e2f116-07af-4358-b44d-e7d1429dd345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de8e4142bed424d8845afcd9dbeaf01', 'request_id': 'req-42e2f116-07af-4358-b44d-e7d1429dd345'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:21.180300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-42e2f116-07af-4358-b44d-e7d1429dd345 admin admin] The token cache is being invalidate because project 0de8e4142bed424d8845afcd9dbeaf01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:21.180613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-42e2f116-07af-4358-b44d-e7d1429dd345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:21.181267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-42e2f116-07af-4358-b44d-e7d1429dd345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:21.182798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 928/1866] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:21 2022] DELETE /identity/v3/projects/0de8e4142bed424d8845afcd9dbeaf01 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:21.187332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6515b-394a-4bfd-b9fa-644fe68de933 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.187617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6515b-394a-4bfd-b9fa-644fe68de933 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.187892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6515b-394a-4bfd-b9fa-644fe68de933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.205211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-cbc6515b-394a-4bfd-b9fa-644fe68de933 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:21.206386 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-cbc6515b-394a-4bfd-b9fa-644fe68de933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:21.230757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 939/1867] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:20:21.242602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2a80d-a9c6-48bf-8744-9e2151de578a None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:21.244089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-21d2a80d-a9c6-48bf-8744-9e2151de578a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:21.267076 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 929/1868] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:20:21.275234 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ba5e3-5892-4dec-a261-d7cbc772ea67 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.275530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ba5e3-5892-4dec-a261-d7cbc772ea67 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.275812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ba5e3-5892-4dec-a261-d7cbc772ea67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.294368 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c82ba5e3-5892-4dec-a261-d7cbc772ea67 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:21.295582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c82ba5e3-5892-4dec-a261-d7cbc772ea67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:21.303799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 930/1869] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:20:21.307949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ba5e3-5892-4dec-a261-d7cbc772ea67 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:21.309585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2a80d-a9c6-48bf-8744-9e2151de578a None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:21.313447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c63a2-1b38-4373-83b7-d839172961e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:21.314221 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c63a2-1b38-4373-83b7-d839172961e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.314459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c63a2-1b38-4373-83b7-d839172961e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.314740 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c63a2-1b38-4373-83b7-d839172961e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.318910 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ba5e3-5892-4dec-a261-d7cbc772ea67 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:21.324488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c18b5-297f-463c-9e3e-363684350454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:21.325150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c18b5-297f-463c-9e3e-363684350454 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.325434 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c18b5-297f-463c-9e3e-363684350454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.325571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c63a2-1b38-4373-83b7-d839172961e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:21.325867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c18b5-297f-463c-9e3e-363684350454 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.327067 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c18b5-297f-463c-9e3e-363684350454 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:21.327377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c18b5-297f-463c-9e3e-363684350454 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:21.327493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c63a2-1b38-4373-83b7-d839172961e2 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:21.329930 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c18b5-297f-463c-9e3e-363684350454 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:21.351221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 931/1870] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:20:21.356425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c18b5-297f-463c-9e3e-363684350454 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:21.356678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 940/1871] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:20:21.381015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c63a2-1b38-4373-83b7-d839172961e2 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:21.384220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c18b5-297f-463c-9e3e-363684350454 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:21.395899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-129c63a2-1b38-4373-83b7-d839172961e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:21.396055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127df02b-1a7a-4a2a-9d18-0b0e82b4a5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:21.396743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127df02b-1a7a-4a2a-9d18-0b0e82b4a5eb admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.397042 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127df02b-1a7a-4a2a-9d18-0b0e82b4a5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.397465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127df02b-1a7a-4a2a-9d18-0b0e82b4a5eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.398169 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127df02b-1a7a-4a2a-9d18-0b0e82b4a5eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:21.402814 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127df02b-1a7a-4a2a-9d18-0b0e82b4a5eb admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:21.417032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c63a2-1b38-4373-83b7-d839172961e2 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:21.423018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a189f2f-5320-4959-b48e-081c3f3f7a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:21.424000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a189f2f-5320-4959-b48e-081c3f3f7a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.424438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a189f2f-5320-4959-b48e-081c3f3f7a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.424909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a189f2f-5320-4959-b48e-081c3f3f7a2e admin admin] PATH_INFO: `/v3/users/1575eb17ca494619bf67c7c8b4ec94a7` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.426714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a189f2f-5320-4959-b48e-081c3f3f7a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1575eb17ca494619bf67c7c8b4ec94a7)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:21.428895 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a189f2f-5320-4959-b48e-081c3f3f7a2e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:21.443395 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:21.449848 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 932/1872] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:21 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:21.455229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127df02b-1a7a-4a2a-9d18-0b0e82b4a5eb admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:21.525430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0a189f2f-5320-4959-b48e-081c3f3f7a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1575eb17ca494619bf67c7c8b4ec94a7'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:21.529079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:21.536539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127df02b-1a7a-4a2a-9d18-0b0e82b4a5eb admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:21.541957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0a189f2f-5320-4959-b48e-081c3f3f7a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1575eb17ca494619bf67c7c8b4ec94a7'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:21.542928 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0a189f2f-5320-4959-b48e-081c3f3f7a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1575eb17ca494619bf67c7c8b4ec94a7'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:21.546196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b07b72-a6d5-4dac-b591-23ed0fb7ca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:21.547296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b07b72-a6d5-4dac-b591-23ed0fb7ca87 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.547887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b07b72-a6d5-4dac-b591-23ed0fb7ca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.548403 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b07b72-a6d5-4dac-b591-23ed0fb7ca87 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.550030 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b07b72-a6d5-4dac-b591-23ed0fb7ca87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:21.553036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b07b72-a6d5-4dac-b591-23ed0fb7ca87 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:21.574548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0a189f2f-5320-4959-b48e-081c3f3f7a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1575eb17ca494619bf67c7c8b4ec94a7'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:21.575963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0a189f2f-5320-4959-b48e-081c3f3f7a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1575eb17ca494619bf67c7c8b4ec94a7'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:21.591803 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 933/1873] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:21 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:21.596253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b07b72-a6d5-4dac-b591-23ed0fb7ca87 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:21.597227 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 941/1874] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:21 2022] DELETE /identity/v3/users/1575eb17ca494619bf67c7c8b4ec94a7 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:21.601743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac8337-898d-4716-a5cb-68759256256e None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.602696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-47b07b72-a6d5-4dac-b591-23ed0fb7ca87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:21.604336 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac8337-898d-4716-a5cb-68759256256e None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.604638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac8337-898d-4716-a5cb-68759256256e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.638765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b07b72-a6d5-4dac-b591-23ed0fb7ca87 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:21.644624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-8bac8337-898d-4716-a5cb-68759256256e None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:21.646050 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8bac8337-898d-4716-a5cb-68759256256e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:21.737362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 942/1875] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:21.740866 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc39da-d7bb-4ace-b92e-8f214f50ddb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:21.741564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc39da-d7bb-4ace-b92e-8f214f50ddb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.741961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc39da-d7bb-4ace-b92e-8f214f50ddb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.742173 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc39da-d7bb-4ace-b92e-8f214f50ddb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.743486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc39da-d7bb-4ace-b92e-8f214f50ddb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:21.745143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc39da-d7bb-4ace-b92e-8f214f50ddb7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:21.761670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 934/1876] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:21.765343 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac8337-898d-4716-a5cb-68759256256e None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:21.770945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc39da-d7bb-4ace-b92e-8f214f50ddb7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:21.774237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6ecc39da-d7bb-4ace-b92e-8f214f50ddb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:21.780005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac8337-898d-4716-a5cb-68759256256e None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:21.787747 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709b344-c5ee-4a39-954d-6fd778d9471f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:21.789467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709b344-c5ee-4a39-954d-6fd778d9471f admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.790604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709b344-c5ee-4a39-954d-6fd778d9471f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.791043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709b344-c5ee-4a39-954d-6fd778d9471f admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.792100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709b344-c5ee-4a39-954d-6fd778d9471f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:21.795568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709b344-c5ee-4a39-954d-6fd778d9471f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:21.817494 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 943/1877] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:21.821654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709b344-c5ee-4a39-954d-6fd778d9471f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:21.840460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709b344-c5ee-4a39-954d-6fd778d9471f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:21.860007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229d484-05e6-4db9-8fa0-bf2abf8d55ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:21.861903 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229d484-05e6-4db9-8fa0-bf2abf8d55ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.862219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229d484-05e6-4db9-8fa0-bf2abf8d55ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.863390 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229d484-05e6-4db9-8fa0-bf2abf8d55ea admin admin] PATH_INFO: `/v3/projects/63e10514ff4b4e69a68e6a3e27f1b91b/users/e3bcaf5f2f41422592e8616bedb6287d/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.891547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229d484-05e6-4db9-8fa0-bf2abf8d55ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e10514ff4b4e69a68e6a3e27f1b91b, user_id=e3bcaf5f2f41422592e8616bedb6287d, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:21.895167 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229d484-05e6-4db9-8fa0-bf2abf8d55ea admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:21.902745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc39da-d7bb-4ace-b92e-8f214f50ddb7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:21.907864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb8a98-a298-4572-a801-4a7d1a1a6d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:21.909211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8a98-a298-4572-a801-4a7d1a1a6d0a service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.909554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8a98-a298-4572-a801-4a7d1a1a6d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.909837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8a98-a298-4572-a801-4a7d1a1a6d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.917565 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 944/1878] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:21 2022] PUT /identity/v3/projects/63e10514ff4b4e69a68e6a3e27f1b91b/users/e3bcaf5f2f41422592e8616bedb6287d/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:21.925123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb8a98-a298-4572-a801-4a7d1a1a6d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:21.927698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb8a98-a298-4572-a801-4a7d1a1a6d0a service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:21.929102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb91bf5-76ea-4459-ad1b-03dd9fb495ce None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:21.929517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb91bf5-76ea-4459-ad1b-03dd9fb495ce None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:21.929674 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb91bf5-76ea-4459-ad1b-03dd9fb495ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:21.941251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 935/1879] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:21.966010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-fbb91bf5-76ea-4459-ad1b-03dd9fb495ce None None] MFA Rules not processed for user `e3bcaf5f2f41422592e8616bedb6287d`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:21.967602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-fbb91bf5-76ea-4459-ad1b-03dd9fb495ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:22.031847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:22.072611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 945/1880] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:22.074346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb8a98-a298-4572-a801-4a7d1a1a6d0a service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:22.084760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb8a98-a298-4572-a801-4a7d1a1a6d0a service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:22.167125 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77b6e6-a74d-4bcb-a4cb-d46a3780e6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:22.168994 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77b6e6-a74d-4bcb-a4cb-d46a3780e6e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:22.169576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77b6e6-a74d-4bcb-a4cb-d46a3780e6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:22.170040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77b6e6-a74d-4bcb-a4cb-d46a3780e6e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:22.171082 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77b6e6-a74d-4bcb-a4cb-d46a3780e6e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:22.173706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77b6e6-a74d-4bcb-a4cb-d46a3780e6e9 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:22.211402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 936/1881] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:22 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:20:22.215403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb91bf5-76ea-4459-ad1b-03dd9fb495ce None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:22.239626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb91bf5-76ea-4459-ad1b-03dd9fb495ce None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:22.358641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733783f-45c6-4dfb-889e-53958995ca0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:22.359928 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733783f-45c6-4dfb-889e-53958995ca0a admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:22.361471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733783f-45c6-4dfb-889e-53958995ca0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:22.362171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733783f-45c6-4dfb-889e-53958995ca0a admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:22.363585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733783f-45c6-4dfb-889e-53958995ca0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:22.366515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733783f-45c6-4dfb-889e-53958995ca0a admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:22.437749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 946/1882] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:22 2022] POST /identity/v3/users => generated 299 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:20:22.443585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77b6e6-a74d-4bcb-a4cb-d46a3780e6e9 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:22.463014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77b6e6-a74d-4bcb-a4cb-d46a3780e6e9 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:22.472306 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d717a-d6ea-40ff-a53a-c46c2495af82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:22.473267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d717a-d6ea-40ff-a53a-c46c2495af82 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:22.473590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d717a-d6ea-40ff-a53a-c46c2495af82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:22.473961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d717a-d6ea-40ff-a53a-c46c2495af82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:22.474702 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d717a-d6ea-40ff-a53a-c46c2495af82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:22.476396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d717a-d6ea-40ff-a53a-c46c2495af82 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:22.481076 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:22.496668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 937/1883] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:22.500970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733783f-45c6-4dfb-889e-53958995ca0a admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:22.517056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733783f-45c6-4dfb-889e-53958995ca0a admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:22.525454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62854a18-b5ef-46b3-a8a3-c1056af5cb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:22.526692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62854a18-b5ef-46b3-a8a3-c1056af5cb3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:22.527576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62854a18-b5ef-46b3-a8a3-c1056af5cb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:22.527576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62854a18-b5ef-46b3-a8a3-c1056af5cb3d admin admin] PATH_INFO: `/v3/projects/50cc17436227484183b977517a2cd94a/users/f48778eca026457286d8cb3e5aa1b6da/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:22.556003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62854a18-b5ef-46b3-a8a3-c1056af5cb3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=50cc17436227484183b977517a2cd94a, user_id=f48778eca026457286d8cb3e5aa1b6da, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:22.559239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62854a18-b5ef-46b3-a8a3-c1056af5cb3d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:22.575869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d717a-d6ea-40ff-a53a-c46c2495af82 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:22.578173 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b10d717a-d6ea-40ff-a53a-c46c2495af82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:22.587987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 947/1884] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:22 2022] PUT /identity/v3/projects/50cc17436227484183b977517a2cd94a/users/f48778eca026457286d8cb3e5aa1b6da/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:22.588741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62854a18-b5ef-46b3-a8a3-c1056af5cb3d admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:22.614347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d717a-d6ea-40ff-a53a-c46c2495af82 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:22.720721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62854a18-b5ef-46b3-a8a3-c1056af5cb3d admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:22.727594 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338c626-5128-4f50-ac23-811ae568d456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:22.728384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338c626-5128-4f50-ac23-811ae568d456 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:22.728697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338c626-5128-4f50-ac23-811ae568d456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:22.728981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338c626-5128-4f50-ac23-811ae568d456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:22.729723 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338c626-5128-4f50-ac23-811ae568d456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:22.731472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338c626-5128-4f50-ac23-811ae568d456 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:22.742968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 948/1885] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:22.747017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338c626-5128-4f50-ac23-811ae568d456 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:22.748323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd78e29-e764-4216-8fbe-013fbe56b077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:22.749302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd78e29-e764-4216-8fbe-013fbe56b077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:22.749500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd78e29-e764-4216-8fbe-013fbe56b077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:22.750070 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd78e29-e764-4216-8fbe-013fbe56b077 admin admin] PATH_INFO: `/v3/users/ce9f0e19d76a440c8b7365fc1449d41b` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:22.751369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd78e29-e764-4216-8fbe-013fbe56b077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9f0e19d76a440c8b7365fc1449d41b)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:22.753348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd78e29-e764-4216-8fbe-013fbe56b077 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:22.759348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338c626-5128-4f50-ac23-811ae568d456 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:22.766924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b98c4-6d62-41ac-a0ab-679c5fdc67f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:22.768236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b98c4-6d62-41ac-a0ab-679c5fdc67f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:22.768420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b98c4-6d62-41ac-a0ab-679c5fdc67f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:22.768701 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b98c4-6d62-41ac-a0ab-679c5fdc67f8 admin admin] PATH_INFO: `/v3/projects/50cc17436227484183b977517a2cd94a/users/f48778eca026457286d8cb3e5aa1b6da/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:22.772943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b98c4-6d62-41ac-a0ab-679c5fdc67f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50cc17436227484183b977517a2cd94a, user_id=f48778eca026457286d8cb3e5aa1b6da, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:22.777211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b98c4-6d62-41ac-a0ab-679c5fdc67f8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:22.795721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 949/1886] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:22 2022] PUT /identity/v3/projects/50cc17436227484183b977517a2cd94a/users/f48778eca026457286d8cb3e5aa1b6da/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:22.808630 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5a24f-3cdb-4312-9b87-ab4fbe0a94a4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:22.809496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5a24f-3cdb-4312-9b87-ab4fbe0a94a4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:22.809496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5a24f-3cdb-4312-9b87-ab4fbe0a94a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:22.858341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-b3f5a24f-3cdb-4312-9b87-ab4fbe0a94a4 None None] MFA Rules not processed for user `f48778eca026457286d8cb3e5aa1b6da`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:22.859794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b3f5a24f-3cdb-4312-9b87-ab4fbe0a94a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:22.860680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9bd78e29-e764-4216-8fbe-013fbe56b077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9f0e19d76a440c8b7365fc1449d41b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:22.861620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9bd78e29-e764-4216-8fbe-013fbe56b077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9f0e19d76a440c8b7365fc1449d41b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:22.875564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9bd78e29-e764-4216-8fbe-013fbe56b077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9f0e19d76a440c8b7365fc1449d41b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:22.886844 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:22.903918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9bd78e29-e764-4216-8fbe-013fbe56b077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9f0e19d76a440c8b7365fc1449d41b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:22.904908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9bd78e29-e764-4216-8fbe-013fbe56b077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9f0e19d76a440c8b7365fc1449d41b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:22.925206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 938/1887] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:22 2022] DELETE /identity/v3/users/ce9f0e19d76a440c8b7365fc1449d41b => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:22.930217 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff51002-2b57-4545-b573-bc5979b18059 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:22.930599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff51002-2b57-4545-b573-bc5979b18059 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:22.930958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff51002-2b57-4545-b573-bc5979b18059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:22.952550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-1ff51002-2b57-4545-b573-bc5979b18059 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:22.953775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-1ff51002-2b57-4545-b573-bc5979b18059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:22.968420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 950/1888] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:22.979623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5a24f-3cdb-4312-9b87-ab4fbe0a94a4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:22.982428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b3f5a24f-3cdb-4312-9b87-ab4fbe0a94a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:23.028831 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 939/1889] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:23.039329 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff51002-2b57-4545-b573-bc5979b18059 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:23.041781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-1ff51002-2b57-4545-b573-bc5979b18059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:23.044885 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5a24f-3cdb-4312-9b87-ab4fbe0a94a4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:23.062594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:23.106421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1757020-70da-48b7-a4e9-0ada6aacb6a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:23.107091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757020-70da-48b7-a4e9-0ada6aacb6a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.107386 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757020-70da-48b7-a4e9-0ada6aacb6a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.107671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757020-70da-48b7-a4e9-0ada6aacb6a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.110114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff51002-2b57-4545-b573-bc5979b18059 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:23.114054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531390c-fdf5-4fe3-9887-42d442e28196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:23.114769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531390c-fdf5-4fe3-9887-42d442e28196 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.115057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531390c-fdf5-4fe3-9887-42d442e28196 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.115335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531390c-fdf5-4fe3-9887-42d442e28196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.125744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531390c-fdf5-4fe3-9887-42d442e28196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:23.127166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531390c-fdf5-4fe3-9887-42d442e28196 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:23.137489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 940/1890] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:20:23.145061 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531390c-fdf5-4fe3-9887-42d442e28196 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:23.146354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-2531390c-fdf5-4fe3-9887-42d442e28196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:23.159772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531390c-fdf5-4fe3-9887-42d442e28196 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:23.175377 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1757020-70da-48b7-a4e9-0ada6aacb6a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:23.175681 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:23.176648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.177035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.177350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8 admin admin] PATH_INFO: `/v3/projects/8ac76d0685db4a5cac0605892884deaf` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.177452 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1757020-70da-48b7-a4e9-0ada6aacb6a0 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:23.178672 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac76d0685db4a5cac0605892884deaf)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:23.181724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:23.190231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 951/1891] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:23.224994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1757020-70da-48b7-a4e9-0ada6aacb6a0 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:23.226471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b1757020-70da-48b7-a4e9-0ada6aacb6a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:23.247540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1757020-70da-48b7-a4e9-0ada6aacb6a0 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:23.259580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac76d0685db4a5cac0605892884deaf', 'request_id': 'req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.267620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac76d0685db4a5cac0605892884deaf', 'request_id': 'req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.270816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:23.279016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac76d0685db4a5cac0605892884deaf', 'request_id': 'req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.293193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8 admin admin] The token cache is being invalidate because project 8ac76d0685db4a5cac0605892884deaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:23.293488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.294842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e0c0fb5-449e-4ef7-b5cd-4318f9b9c6c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.299975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 941/1892] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:23 2022] DELETE /identity/v3/projects/8ac76d0685db4a5cac0605892884deaf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:23.301747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff23c9e-2f6b-44f9-84f7-a99ad9ab6dee None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.303518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff23c9e-2f6b-44f9-84f7-a99ad9ab6dee None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.303904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff23c9e-2f6b-44f9-84f7-a99ad9ab6dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.327329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64a566-bbd0-422a-b59d-515bd3016651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:23.328999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64a566-bbd0-422a-b59d-515bd3016651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.329432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64a566-bbd0-422a-b59d-515bd3016651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.329928 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64a566-bbd0-422a-b59d-515bd3016651 admin admin] PATH_INFO: `/v3/users/e3bcaf5f2f41422592e8616bedb6287d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.331679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64a566-bbd0-422a-b59d-515bd3016651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3bcaf5f2f41422592e8616bedb6287d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:23.333006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-4ff23c9e-2f6b-44f9-84f7-a99ad9ab6dee None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:23.334345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64a566-bbd0-422a-b59d-515bd3016651 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:23.335011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-4ff23c9e-2f6b-44f9-84f7-a99ad9ab6dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:23.347166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 942/1893] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:20:23.357977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15bf88-0c9d-4e4c-95cb-674b49dd05e0 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.358268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15bf88-0c9d-4e4c-95cb-674b49dd05e0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.358550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15bf88-0c9d-4e4c-95cb-674b49dd05e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.399677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1e64a566-bbd0-422a-b59d-515bd3016651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3bcaf5f2f41422592e8616bedb6287d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.411240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1e64a566-bbd0-422a-b59d-515bd3016651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3bcaf5f2f41422592e8616bedb6287d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.412357 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1e64a566-bbd0-422a-b59d-515bd3016651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3bcaf5f2f41422592e8616bedb6287d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.422004 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3e15bf88-0c9d-4e4c-95cb-674b49dd05e0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:23.423773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3e15bf88-0c9d-4e4c-95cb-674b49dd05e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:23.430613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1e64a566-bbd0-422a-b59d-515bd3016651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3bcaf5f2f41422592e8616bedb6287d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.431517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1e64a566-bbd0-422a-b59d-515bd3016651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3bcaf5f2f41422592e8616bedb6287d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.441889 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 943/1894] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:23.442842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15bf88-0c9d-4e4c-95cb-674b49dd05e0 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:23.446192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 952/1895] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:23 2022] DELETE /identity/v3/users/e3bcaf5f2f41422592e8616bedb6287d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:23.450524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92cd98-c919-4dcc-89ed-b68eca7153cd None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.450708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92cd98-c919-4dcc-89ed-b68eca7153cd None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.451000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92cd98-c919-4dcc-89ed-b68eca7153cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.455686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15bf88-0c9d-4e4c-95cb-674b49dd05e0 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:23.479161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-cb92cd98-c919-4dcc-89ed-b68eca7153cd None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:23.480580 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cb92cd98-c919-4dcc-89ed-b68eca7153cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:23.535348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dedb0d-51c2-4170-bce4-6ed434f4a115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:23.536610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dedb0d-51c2-4170-bce4-6ed434f4a115 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.536848 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dedb0d-51c2-4170-bce4-6ed434f4a115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.537236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dedb0d-51c2-4170-bce4-6ed434f4a115 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.538050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dedb0d-51c2-4170-bce4-6ed434f4a115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:23.538420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dedb0d-51c2-4170-bce4-6ed434f4a115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:23.540776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dedb0d-51c2-4170-bce4-6ed434f4a115 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:23.547751 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 953/1896] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:23.557032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 944/1897] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:20:23.557210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92cd98-c919-4dcc-89ed-b68eca7153cd None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:23.557729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cb92cd98-c919-4dcc-89ed-b68eca7153cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:23.558542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dedb0d-51c2-4170-bce4-6ed434f4a115 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:23.560493 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-86dedb0d-51c2-4170-bce4-6ed434f4a115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:23.580195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dedb0d-51c2-4170-bce4-6ed434f4a115 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:23.596095 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49a201-6c15-4e6c-a322-a73b4ae36dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:23.597009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49a201-6c15-4e6c-a322-a73b4ae36dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.597339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49a201-6c15-4e6c-a322-a73b4ae36dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.597692 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49a201-6c15-4e6c-a322-a73b4ae36dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.598439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49a201-6c15-4e6c-a322-a73b4ae36dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:23.601022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49a201-6c15-4e6c-a322-a73b4ae36dc0 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:23.616516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:23.632117 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 945/1898] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:23 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:20:23.632392 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49a201-6c15-4e6c-a322-a73b4ae36dc0 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:23.635037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92cd98-c919-4dcc-89ed-b68eca7153cd None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:23.636108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-de49a201-6c15-4e6c-a322-a73b4ae36dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:23.640957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:23.694853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90edf5-ef17-47cc-96a0-3378a6a1ac9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:23.695534 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90edf5-ef17-47cc-96a0-3378a6a1ac9a service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.696378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90edf5-ef17-47cc-96a0-3378a6a1ac9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.696668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90edf5-ef17-47cc-96a0-3378a6a1ac9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.722432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49a201-6c15-4e6c-a322-a73b4ae36dc0 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:23.731093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:23.732137 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.732556 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.732934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] PATH_INFO: `/v3/projects/7fcb281c60c442ff9181f1993959d8c3` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.734288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fcb281c60c442ff9181f1993959d8c3)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:23.736332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:23.765774 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90edf5-ef17-47cc-96a0-3378a6a1ac9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:23.767300 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90edf5-ef17-47cc-96a0-3378a6a1ac9a service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:23.777888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 954/1899] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:23.778732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90edf5-ef17-47cc-96a0-3378a6a1ac9a service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:23.786229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fcb281c60c442ff9181f1993959d8c3', 'request_id': 'req-b97e27f2-f58c-465e-b491-cbf103e6e6f4'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.788406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90edf5-ef17-47cc-96a0-3378a6a1ac9a service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:23.795444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fcb281c60c442ff9181f1993959d8c3', 'request_id': 'req-b97e27f2-f58c-465e-b491-cbf103e6e6f4'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.804363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fcb281c60c442ff9181f1993959d8c3', 'request_id': 'req-b97e27f2-f58c-465e-b491-cbf103e6e6f4'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.816568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] The token cache is being invalidate because project 7fcb281c60c442ff9181f1993959d8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:23.817008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.817701 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:23.819695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 946/1900] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:23 2022] DELETE /identity/v3/projects/7fcb281c60c442ff9181f1993959d8c3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:23.824096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:23.825599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:23.860798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c102d34-4ea5-4793-83d2-18b53d139919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:23.862102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c102d34-4ea5-4793-83d2-18b53d139919 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.862568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c102d34-4ea5-4793-83d2-18b53d139919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.862944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c102d34-4ea5-4793-83d2-18b53d139919 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.864053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c102d34-4ea5-4793-83d2-18b53d139919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:23.866515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c102d34-4ea5-4793-83d2-18b53d139919 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:23.893677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 955/1901] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:23 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:23.897517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c102d34-4ea5-4793-83d2-18b53d139919 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:23.900402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-9c102d34-4ea5-4793-83d2-18b53d139919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:23.916424 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c102d34-4ea5-4793-83d2-18b53d139919 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:23.923504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78c4e2-f8b9-464d-b2fe-0736e4a421c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:23.924234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78c4e2-f8b9-464d-b2fe-0736e4a421c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.924663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78c4e2-f8b9-464d-b2fe-0736e4a421c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.924953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78c4e2-f8b9-464d-b2fe-0736e4a421c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.925693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78c4e2-f8b9-464d-b2fe-0736e4a421c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:23.927955 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78c4e2-f8b9-464d-b2fe-0736e4a421c5 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:23.935264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e27f2-f58c-465e-b491-cbf103e6e6f4 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:23.940993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:23.942131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.942594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.942996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] PATH_INFO: `/v3/users/cb1c6bfff18c48a5897483e0572213dc` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.944821 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb1c6bfff18c48a5897483e0572213dc)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:23.948869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:23.965085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 956/1902] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:23.968961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78c4e2-f8b9-464d-b2fe-0736e4a421c5 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:23.986431 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78c4e2-f8b9-464d-b2fe-0736e4a421c5 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:23.991625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b92911-429c-40a9-b3b2-9506554dfec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:23.992481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b92911-429c-40a9-b3b2-9506554dfec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:23.992899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b92911-429c-40a9-b3b2-9506554dfec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:23.993286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b92911-429c-40a9-b3b2-9506554dfec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:23.994123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b92911-429c-40a9-b3b2-9506554dfec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:23.996374 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b92911-429c-40a9-b3b2-9506554dfec3 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:24.016142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 957/1903] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:24.018112 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b92911-429c-40a9-b3b2-9506554dfec3 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:24.024992 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:24.035264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b92911-429c-40a9-b3b2-9506554dfec3 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:24.041373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb1c6bfff18c48a5897483e0572213dc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.043263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb1c6bfff18c48a5897483e0572213dc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.046042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5e1bc-89a6-40e1-8b50-adf5a273531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:24.047230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5e1bc-89a6-40e1-8b50-adf5a273531b admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:24.047599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5e1bc-89a6-40e1-8b50-adf5a273531b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:24.047893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5e1bc-89a6-40e1-8b50-adf5a273531b admin admin] PATH_INFO: `/v3/projects/cabdc25e41ac45ae824567f3b2aca39c/users/d8e2b6536c48498db1c3cf97b141ab6f/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:24.053968 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb1c6bfff18c48a5897483e0572213dc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.063559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:24.073433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5e1bc-89a6-40e1-8b50-adf5a273531b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cabdc25e41ac45ae824567f3b2aca39c, user_id=d8e2b6536c48498db1c3cf97b141ab6f, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:24.078761 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5e1bc-89a6-40e1-8b50-adf5a273531b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:24.084581 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb1c6bfff18c48a5897483e0572213dc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.085697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb1c6bfff18c48a5897483e0572213dc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.097311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 958/1904] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:24 2022] PUT /identity/v3/projects/cabdc25e41ac45ae824567f3b2aca39c/users/d8e2b6536c48498db1c3cf97b141ab6f/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:24.109535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34d3da-253e-469c-a9fe-1a2314113e20 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:24.111267 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34d3da-253e-469c-a9fe-1a2314113e20 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:24.111518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34d3da-253e-469c-a9fe-1a2314113e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:24.120108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 947/1905] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:23 2022] DELETE /identity/v3/users/cb1c6bfff18c48a5897483e0572213dc => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:24.153744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-7e34d3da-253e-469c-a9fe-1a2314113e20 None None] MFA Rules not processed for user `d8e2b6536c48498db1c3cf97b141ab6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:24.155907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7e34d3da-253e-469c-a9fe-1a2314113e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:24.261247 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 959/1906] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:24.272527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:24.274044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:24.347433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a99d79-48da-4ac5-b039-7f89b35783ee admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:24.352044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ec05d-2f2b-4d08-a3ac-b95e9441ace2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48586ff1269f49a3ae3d9f3e40c1b9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:24.353483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ec05d-2f2b-4d08-a3ac-b95e9441ace2 service glance] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:24.353947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ec05d-2f2b-4d08-a3ac-b95e9441ace2 service glance] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:24.354236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ec05d-2f2b-4d08-a3ac-b95e9441ace2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:24.365734 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ec05d-2f2b-4d08-a3ac-b95e9441ace2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:24.367836 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ec05d-2f2b-4d08-a3ac-b95e9441ace2 service glance] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:24.368420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34d3da-253e-469c-a9fe-1a2314113e20 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:24.370616 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7e34d3da-253e-469c-a9fe-1a2314113e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:24.379287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 948/1907] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:24.473472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34d3da-253e-469c-a9fe-1a2314113e20 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:24.474387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ec05d-2f2b-4d08-a3ac-b95e9441ace2 service glance] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:24.476371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-435ec05d-2f2b-4d08-a3ac-b95e9441ace2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:24.479255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:24.480262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:24.480554 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:24.480849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] PATH_INFO: `/v3/projects/63e10514ff4b4e69a68e6a3e27f1b91b` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:24.487504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e10514ff4b4e69a68e6a3e27f1b91b)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:24.495486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:24.514508 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ec05d-2f2b-4d08-a3ac-b95e9441ace2 service glance] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:24.520436 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:24.522368 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:24.522750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:24.523125 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2 admin admin] PATH_INFO: `/v3/projects/9e7e103180e34d0fa958d0c1b3cd62ef` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:24.525916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e7e103180e34d0fa958d0c1b3cd62ef)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:24.528270 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:24.567579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e10514ff4b4e69a68e6a3e27f1b91b', 'request_id': 'req-d2961453-c594-4f82-820a-1eed6ce72a7c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.574749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e10514ff4b4e69a68e6a3e27f1b91b', 'request_id': 'req-d2961453-c594-4f82-820a-1eed6ce72a7c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.582502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e10514ff4b4e69a68e6a3e27f1b91b', 'request_id': 'req-d2961453-c594-4f82-820a-1eed6ce72a7c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.603714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] The token cache is being invalidate because project 63e10514ff4b4e69a68e6a3e27f1b91b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:24.604003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.604943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.606854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 960/1908] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:24 2022] DELETE /identity/v3/projects/63e10514ff4b4e69a68e6a3e27f1b91b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:24.613550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:24.615884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:24.631980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e7e103180e34d0fa958d0c1b3cd62ef', 'request_id': 'req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.633879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:24.648723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e7e103180e34d0fa958d0c1b3cd62ef', 'request_id': 'req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.660976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e7e103180e34d0fa958d0c1b3cd62ef', 'request_id': 'req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.688206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2 admin admin] The token cache is being invalidate because project 9e7e103180e34d0fa958d0c1b3cd62ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:24.688505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.689216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f226aee7-df17-4b05-9e70-8c3c3d2a90d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.691279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 949/1909] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:24 2022] DELETE /identity/v3/projects/9e7e103180e34d0fa958d0c1b3cd62ef => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:24.695527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80936b-ae77-427d-9d96-113e1dcf201d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:24.696430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80936b-ae77-427d-9d96-113e1dcf201d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:24.696785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80936b-ae77-427d-9d96-113e1dcf201d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:24.718401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2961453-c594-4f82-820a-1eed6ce72a7c admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:24.722466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-9e80936b-ae77-427d-9d96-113e1dcf201d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:24.724501 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9e80936b-ae77-427d-9d96-113e1dcf201d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:24.768610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:24.801357 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:24.802668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:24.803065 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:24.803708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 950/1910] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:24.803850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] PATH_INFO: `/v3/users/f48778eca026457286d8cb3e5aa1b6da` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:24.805701 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f48778eca026457286d8cb3e5aa1b6da)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:24.807772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:24.812973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305b4f5-9340-442b-ae5e-608f66e95922 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:24.813264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305b4f5-9340-442b-ae5e-608f66e95922 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:24.813552 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305b4f5-9340-442b-ae5e-608f66e95922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:24.833934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-7305b4f5-9340-442b-ae5e-608f66e95922 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:24.835162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7305b4f5-9340-442b-ae5e-608f66e95922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:24.844580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 951/1911] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:24.849571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305b4f5-9340-442b-ae5e-608f66e95922 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:24.864289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305b4f5-9340-442b-ae5e-608f66e95922 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:24.869100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f48778eca026457286d8cb3e5aa1b6da'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.870799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c160cf-89b5-496f-b06a-39bc62b8e8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:24.872202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c160cf-89b5-496f-b06a-39bc62b8e8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:24.872423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c160cf-89b5-496f-b06a-39bc62b8e8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:24.872806 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c160cf-89b5-496f-b06a-39bc62b8e8e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:24.873557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c160cf-89b5-496f-b06a-39bc62b8e8e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:24.873883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c160cf-89b5-496f-b06a-39bc62b8e8e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:24.875730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c160cf-89b5-496f-b06a-39bc62b8e8e7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:24.878952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f48778eca026457286d8cb3e5aa1b6da'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.879990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f48778eca026457286d8cb3e5aa1b6da'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.890055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 952/1912] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:20:24.896403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f48778eca026457286d8cb3e5aa1b6da'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.897048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f48778eca026457286d8cb3e5aa1b6da'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:24.897747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c160cf-89b5-496f-b06a-39bc62b8e8e7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:24.900298 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-54c160cf-89b5-496f-b06a-39bc62b8e8e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:24.914276 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 961/1913] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:24 2022] DELETE /identity/v3/users/f48778eca026457286d8cb3e5aa1b6da => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:24.919726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c160cf-89b5-496f-b06a-39bc62b8e8e7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:24.996704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:24.998463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:25.000708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b85be1-fc3f-4bf3-8b0b-44bce733e77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:25.001385 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b85be1-fc3f-4bf3-8b0b-44bce733e77e admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.001673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b85be1-fc3f-4bf3-8b0b-44bce733e77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.001952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b85be1-fc3f-4bf3-8b0b-44bce733e77e admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.002787 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b85be1-fc3f-4bf3-8b0b-44bce733e77e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:25.005085 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b85be1-fc3f-4bf3-8b0b-44bce733e77e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:25.021531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02395b30-fb3c-4fe6-9152-004c092d844a admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:25.040239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 953/1914] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:24 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:25.043213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe3c2a-6ba9-431c-b776-ccabb4f8f911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:25.044138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe3c2a-6ba9-431c-b776-ccabb4f8f911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.044354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b85be1-fc3f-4bf3-8b0b-44bce733e77e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:25.044485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe3c2a-6ba9-431c-b776-ccabb4f8f911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.044736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe3c2a-6ba9-431c-b776-ccabb4f8f911 admin admin] PATH_INFO: `/v3/users/41120c85efc94fe7ab8e14469949df2f` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.045884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe3c2a-6ba9-431c-b776-ccabb4f8f911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41120c85efc94fe7ab8e14469949df2f)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:25.047821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe3c2a-6ba9-431c-b776-ccabb4f8f911 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:25.100902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:25.122626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-57fe3c2a-6ba9-431c-b776-ccabb4f8f911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41120c85efc94fe7ab8e14469949df2f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:25.133480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-57fe3c2a-6ba9-431c-b776-ccabb4f8f911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41120c85efc94fe7ab8e14469949df2f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:25.135093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-57fe3c2a-6ba9-431c-b776-ccabb4f8f911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41120c85efc94fe7ab8e14469949df2f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:25.150799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b85be1-fc3f-4bf3-8b0b-44bce733e77e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:25.153976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-57fe3c2a-6ba9-431c-b776-ccabb4f8f911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41120c85efc94fe7ab8e14469949df2f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:25.154798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-57fe3c2a-6ba9-431c-b776-ccabb4f8f911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41120c85efc94fe7ab8e14469949df2f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:25.157323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf95d0-2af5-454d-9a56-6feb26900eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:25.158263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf95d0-2af5-454d-9a56-6feb26900eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.158704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf95d0-2af5-454d-9a56-6feb26900eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.159136 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf95d0-2af5-454d-9a56-6feb26900eb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.159985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf95d0-2af5-454d-9a56-6feb26900eb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:25.162910 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf95d0-2af5-454d-9a56-6feb26900eb3 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:25.170426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 962/1915] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:24 2022] DELETE /identity/v3/users/41120c85efc94fe7ab8e14469949df2f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:25.175087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef434ab-ee3d-4a88-be28-53ba896be00d None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.175597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef434ab-ee3d-4a88-be28-53ba896be00d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.176038 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef434ab-ee3d-4a88-be28-53ba896be00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.191452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 954/1916] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:25 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:25.192624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf95d0-2af5-454d-9a56-6feb26900eb3 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:25.194264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-91cf95d0-2af5-454d-9a56-6feb26900eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:25.199668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-8ef434ab-ee3d-4a88-be28-53ba896be00d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:25.201055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8ef434ab-ee3d-4a88-be28-53ba896be00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:25.206315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf95d0-2af5-454d-9a56-6feb26900eb3 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:25.241505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:25.278831 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c68a4-2b80-4a85-a349-4c43495dc7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:25.279786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c68a4-2b80-4a85-a349-4c43495dc7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.280215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c68a4-2b80-4a85-a349-4c43495dc7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.280552 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c68a4-2b80-4a85-a349-4c43495dc7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.281541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c68a4-2b80-4a85-a349-4c43495dc7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:25.284372 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c68a4-2b80-4a85-a349-4c43495dc7a5 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:25.302080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 955/1917] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:25.303174 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c68a4-2b80-4a85-a349-4c43495dc7a5 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:25.315077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c68a4-2b80-4a85-a349-4c43495dc7a5 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:25.324027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb0ad7-9d9a-4a41-9524-e11faad730e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:25.325089 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb0ad7-9d9a-4a41-9524-e11faad730e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.325483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb0ad7-9d9a-4a41-9524-e11faad730e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.325898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb0ad7-9d9a-4a41-9524-e11faad730e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.327074 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb0ad7-9d9a-4a41-9524-e11faad730e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:25.329188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb0ad7-9d9a-4a41-9524-e11faad730e8 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:25.347543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 956/1918] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:25.348624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb0ad7-9d9a-4a41-9524-e11faad730e8 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:25.351617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-84bb0ad7-9d9a-4a41-9524-e11faad730e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:25.375951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb0ad7-9d9a-4a41-9524-e11faad730e8 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:25.395486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:25.396908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.397298 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.397682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] PATH_INFO: `/v3/projects/50cc17436227484183b977517a2cd94a` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.399732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50cc17436227484183b977517a2cd94a)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:25.403721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:25.476105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 963/1919] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:20:25.476105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef434ab-ee3d-4a88-be28-53ba896be00d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:25.502458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50cc17436227484183b977517a2cd94a', 'request_id': 'req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:25.506861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef434ab-ee3d-4a88-be28-53ba896be00d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:25.526366 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50cc17436227484183b977517a2cd94a', 'request_id': 'req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:25.535691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50cc17436227484183b977517a2cd94a', 'request_id': 'req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:25.549676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] The token cache is being invalidate because project 50cc17436227484183b977517a2cd94a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:25.550006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:25.550773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:25.553580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 957/1920] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:25 2022] DELETE /identity/v3/projects/50cc17436227484183b977517a2cd94a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:25.556943 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:25.577651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:25.652493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c46c9-3796-433a-984b-bd9545a55353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:25.654227 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c46c9-3796-433a-984b-bd9545a55353 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.654698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c46c9-3796-433a-984b-bd9545a55353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.655101 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c46c9-3796-433a-984b-bd9545a55353 admin admin] PATH_INFO: `/v3/projects/082383cd98734e94bc8ba7275ea2fdff/users/1fdb7235f8164c49b9559ceb491ba47d/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.673558 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c2b39-1ebf-4c5e-81fd-5d7ac9c54b21 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:25.682746 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81179f92-17cb-4642-b1f8-13d438e99eb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:25.684042 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81179f92-17cb-4642-b1f8-13d438e99eb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.684549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81179f92-17cb-4642-b1f8-13d438e99eb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.684974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81179f92-17cb-4642-b1f8-13d438e99eb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.691014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c46c9-3796-433a-984b-bd9545a55353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=082383cd98734e94bc8ba7275ea2fdff, user_id=1fdb7235f8164c49b9559ceb491ba47d, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:25.694359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c46c9-3796-433a-984b-bd9545a55353 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:25.695106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-81179f92-17cb-4642-b1f8-13d438e99eb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:25.711316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81179f92-17cb-4642-b1f8-13d438e99eb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:25.714126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81179f92-17cb-4642-b1f8-13d438e99eb2 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:25.723702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 964/1921] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:25 2022] PUT /identity/v3/projects/082383cd98734e94bc8ba7275ea2fdff/users/1fdb7235f8164c49b9559ceb491ba47d/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:25.725778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4c36f-fa71-4652-b757-6844f166dbca None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.726082 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4c36f-fa71-4652-b757-6844f166dbca None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.726332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 958/1922] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:25.726456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4c36f-fa71-4652-b757-6844f166dbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.728335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922fd4-c788-4684-84d6-7e98da8e7ebb None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.728732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922fd4-c788-4684-84d6-7e98da8e7ebb None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.729103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922fd4-c788-4684-84d6-7e98da8e7ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.758401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-27b4c36f-fa71-4652-b757-6844f166dbca None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:25.759655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-27b4c36f-fa71-4652-b757-6844f166dbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:25.768530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-29922fd4-c788-4684-84d6-7e98da8e7ebb None None] MFA Rules not processed for user `1fdb7235f8164c49b9559ceb491ba47d`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:25.771831 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-29922fd4-c788-4684-84d6-7e98da8e7ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:25.794692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:25.823039 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:25.843759 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 965/1923] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:25.855946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c82037-3b8d-41c2-8c90-031ae0a383c4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.856400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c82037-3b8d-41c2-8c90-031ae0a383c4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.857073 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c82037-3b8d-41c2-8c90-031ae0a383c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.883064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c6c82037-3b8d-41c2-8c90-031ae0a383c4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:25.885267 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c6c82037-3b8d-41c2-8c90-031ae0a383c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:25.903636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 966/1924] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:25.908033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c82037-3b8d-41c2-8c90-031ae0a383c4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:25.909237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 959/1925] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:25.914231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922fd4-c788-4684-84d6-7e98da8e7ebb None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:25.917875 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-29922fd4-c788-4684-84d6-7e98da8e7ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:25.944570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922fd4-c788-4684-84d6-7e98da8e7ebb None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:25.950458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c82037-3b8d-41c2-8c90-031ae0a383c4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:25.961196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6e680-12d9-42a9-a87f-ab44e34d7239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:25.962679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6e680-12d9-42a9-a87f-ab44e34d7239 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.964349 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c212fbc-6f06-4093-90b3-0a471d23bffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:25.965031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c212fbc-6f06-4093-90b3-0a471d23bffb admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:25.965453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c212fbc-6f06-4093-90b3-0a471d23bffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.965735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c212fbc-6f06-4093-90b3-0a471d23bffb admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.966424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c212fbc-6f06-4093-90b3-0a471d23bffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:25.968385 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c212fbc-6f06-4093-90b3-0a471d23bffb admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:25.968660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6e680-12d9-42a9-a87f-ab44e34d7239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:25.969051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6e680-12d9-42a9-a87f-ab44e34d7239 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:25.970018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6e680-12d9-42a9-a87f-ab44e34d7239 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:25.971405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6e680-12d9-42a9-a87f-ab44e34d7239 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:25.974956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6e680-12d9-42a9-a87f-ab44e34d7239 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:26.001605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 967/1926] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:20:26.003562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6e680-12d9-42a9-a87f-ab44e34d7239 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:26.004669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 960/1927] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:25 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:26.005577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c212fbc-6f06-4093-90b3-0a471d23bffb admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:26.017359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6e680-12d9-42a9-a87f-ab44e34d7239 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:26.132870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bb0a1-378a-44c2-ac6d-c84050e6f469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:26.133607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bb0a1-378a-44c2-ac6d-c84050e6f469 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:26.133902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bb0a1-378a-44c2-ac6d-c84050e6f469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:26.135859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bb0a1-378a-44c2-ac6d-c84050e6f469 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:26.136622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bb0a1-378a-44c2-ac6d-c84050e6f469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:26.147169 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bb0a1-378a-44c2-ac6d-c84050e6f469 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:26.147259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c212fbc-6f06-4093-90b3-0a471d23bffb admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:26.162781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a12dc-4341-4aec-8e05-234a9d74836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:26.173099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a12dc-4341-4aec-8e05-234a9d74836a admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:26.173099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a12dc-4341-4aec-8e05-234a9d74836a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:26.173099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a12dc-4341-4aec-8e05-234a9d74836a admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:26.173099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a12dc-4341-4aec-8e05-234a9d74836a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:26.173099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a12dc-4341-4aec-8e05-234a9d74836a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:26.203928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 961/1928] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:26 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:26.204802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a12dc-4341-4aec-8e05-234a9d74836a admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:26.209435 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 968/1929] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:26 2022] POST /identity/v3/projects => generated 387 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:26.213627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bb0a1-378a-44c2-ac6d-c84050e6f469 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:26.215596 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a12dc-4341-4aec-8e05-234a9d74836a admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:26.313192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4498114-a71c-4a47-a79e-41b604e63218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:26.313997 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4498114-a71c-4a47-a79e-41b604e63218 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:26.314251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4498114-a71c-4a47-a79e-41b604e63218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:26.314777 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4498114-a71c-4a47-a79e-41b604e63218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:26.315287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4498114-a71c-4a47-a79e-41b604e63218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:26.320508 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4498114-a71c-4a47-a79e-41b604e63218 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:26.342557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 962/1930] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:26.343427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4498114-a71c-4a47-a79e-41b604e63218 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:26.345181 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b4498114-a71c-4a47-a79e-41b604e63218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:26.348185 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bb0a1-378a-44c2-ac6d-c84050e6f469 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:26.355999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227233f1-d287-47c0-b672-ef0fbca95116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:26.357823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227233f1-d287-47c0-b672-ef0fbca95116 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:26.358478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227233f1-d287-47c0-b672-ef0fbca95116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:26.358966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227233f1-d287-47c0-b672-ef0fbca95116 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:26.359195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4498114-a71c-4a47-a79e-41b604e63218 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:26.360437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227233f1-d287-47c0-b672-ef0fbca95116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:26.363328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227233f1-d287-47c0-b672-ef0fbca95116 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:26.364130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:26.365013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:26.365305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:26.365614 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] PATH_INFO: `/v3/projects/a7a3e4f73f204fb99efd5e62842beb75` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:26.366927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a3e4f73f204fb99efd5e62842beb75)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:26.369615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:26.432823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 969/1931] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:26 2022] POST /identity/v3/users => generated 294 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:20:26.437076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227233f1-d287-47c0-b672-ef0fbca95116 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:26.464508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227233f1-d287-47c0-b672-ef0fbca95116 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:26.475031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e732cd-5d8a-4862-a911-916f840936a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:26.478100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e732cd-5d8a-4862-a911-916f840936a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:26.478394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e732cd-5d8a-4862-a911-916f840936a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:26.478678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e732cd-5d8a-4862-a911-916f840936a0 admin admin] PATH_INFO: `/v3/projects/05545bb5fb46427d9aae6434b651715a/users/ac953a40a80744299eb4a1fff7424da2/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:26.495945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:26.496294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e732cd-5d8a-4862-a911-916f840936a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05545bb5fb46427d9aae6434b651715a, user_id=ac953a40a80744299eb4a1fff7424da2, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:26.499197 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e732cd-5d8a-4862-a911-916f840936a0 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:26.519874 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 970/1932] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:26 2022] PUT /identity/v3/projects/05545bb5fb46427d9aae6434b651715a/users/ac953a40a80744299eb4a1fff7424da2/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:26.521141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e732cd-5d8a-4862-a911-916f840936a0 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:26.529562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:26.542041 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a3e4f73f204fb99efd5e62842beb75', 'request_id': 'req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:26.556665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a3e4f73f204fb99efd5e62842beb75', 'request_id': 'req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:26.566941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a3e4f73f204fb99efd5e62842beb75', 'request_id': 'req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:26.590697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] The token cache is being invalidate because project a7a3e4f73f204fb99efd5e62842beb75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:26.591275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:26.592304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:26.594568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 963/1933] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:26 2022] DELETE /identity/v3/projects/a7a3e4f73f204fb99efd5e62842beb75 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:26.595508 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:26.597797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:26.612645 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e732cd-5d8a-4862-a911-916f840936a0 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:26.621986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06f774-7546-4323-861d-3c26e20e3391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:26.622693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06f774-7546-4323-861d-3c26e20e3391 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:26.623006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06f774-7546-4323-861d-3c26e20e3391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:26.623312 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06f774-7546-4323-861d-3c26e20e3391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:26.624092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06f774-7546-4323-861d-3c26e20e3391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:26.625808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06f774-7546-4323-861d-3c26e20e3391 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:26.643211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 971/1934] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:26.644163 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06f774-7546-4323-861d-3c26e20e3391 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:26.646332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-dd06f774-7546-4323-861d-3c26e20e3391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:26.663035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06f774-7546-4323-861d-3c26e20e3391 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:26.669394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:26.670619 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:26.670907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:26.671188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] PATH_INFO: `/v3/users/cd5d778038c04cc4ac516ee87637d781` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:26.672548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd5d778038c04cc4ac516ee87637d781)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:26.675409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:26.719764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f704b2-7e0f-4e54-944f-dbbb00cf2022 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:26.730374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e53a5e-83b1-40e8-a67b-4ee9d3568593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:26.732673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e53a5e-83b1-40e8-a67b-4ee9d3568593 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:26.733697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e53a5e-83b1-40e8-a67b-4ee9d3568593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:26.733942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e53a5e-83b1-40e8-a67b-4ee9d3568593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:26.734810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e53a5e-83b1-40e8-a67b-4ee9d3568593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:26.736623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e53a5e-83b1-40e8-a67b-4ee9d3568593 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:26.746499 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd5d778038c04cc4ac516ee87637d781'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:26.751477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 964/1935] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:26.752787 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e53a5e-83b1-40e8-a67b-4ee9d3568593 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:26.754889 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e6e53a5e-83b1-40e8-a67b-4ee9d3568593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:26.760186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd5d778038c04cc4ac516ee87637d781'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:26.761236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd5d778038c04cc4ac516ee87637d781'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:26.771123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e53a5e-83b1-40e8-a67b-4ee9d3568593 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:26.780806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd5d778038c04cc4ac516ee87637d781'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:26.781179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75159d10-c5a0-4291-901e-f1ca87d8272b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:26.781982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75159d10-c5a0-4291-901e-f1ca87d8272b admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:26.782375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75159d10-c5a0-4291-901e-f1ca87d8272b admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:26.782671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd5d778038c04cc4ac516ee87637d781'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:26.782860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75159d10-c5a0-4291-901e-f1ca87d8272b admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:26.783756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75159d10-c5a0-4291-901e-f1ca87d8272b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:26.785821 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75159d10-c5a0-4291-901e-f1ca87d8272b admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:26.801163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 965/1936] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:26.801163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75159d10-c5a0-4291-901e-f1ca87d8272b admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:26.801163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-75159d10-c5a0-4291-901e-f1ca87d8272b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:26.807465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 972/1937] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:26 2022] DELETE /identity/v3/users/cd5d778038c04cc4ac516ee87637d781 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:26.808814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:26.810365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:26.818850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75159d10-c5a0-4291-901e-f1ca87d8272b admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:26.907183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fefeac-1406-44a8-bc36-21ff283fc5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:26.908865 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fefeac-1406-44a8-bc36-21ff283fc5e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:26.909427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fefeac-1406-44a8-bc36-21ff283fc5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:26.909724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fefeac-1406-44a8-bc36-21ff283fc5e3 admin admin] PATH_INFO: `/v3/projects/05545bb5fb46427d9aae6434b651715a/users/ac953a40a80744299eb4a1fff7424da2/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:26.913009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fefeac-1406-44a8-bc36-21ff283fc5e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05545bb5fb46427d9aae6434b651715a, user_id=ac953a40a80744299eb4a1fff7424da2, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:26.916021 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fefeac-1406-44a8-bc36-21ff283fc5e3 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:26.933779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cea29-9d71-4519-9f71-ddefe6000144 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:26.937193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 966/1938] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:26 2022] PUT /identity/v3/projects/05545bb5fb46427d9aae6434b651715a/users/ac953a40a80744299eb4a1fff7424da2/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:26.947733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02087b71-d17a-4b72-b7b6-6977926e1699 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:26.948232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02087b71-d17a-4b72-b7b6-6977926e1699 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:26.948588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02087b71-d17a-4b72-b7b6-6977926e1699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:26.962480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:26.995394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-02087b71-d17a-4b72-b7b6-6977926e1699 None None] MFA Rules not processed for user `ac953a40a80744299eb4a1fff7424da2`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:26.997872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-02087b71-d17a-4b72-b7b6-6977926e1699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:27.021602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e013d5-2503-4646-a6f3-66456c3deb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:27.022824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e013d5-2503-4646-a6f3-66456c3deb62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:27.023409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e013d5-2503-4646-a6f3-66456c3deb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:27.023409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e013d5-2503-4646-a6f3-66456c3deb62 admin admin] PATH_INFO: `/v3/projects/16e369c52cb54dbfb5704e70e2c47b00/users/9902a690b63d48989c8325ea0b80ae3c/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:27.047489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:27.052048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e013d5-2503-4646-a6f3-66456c3deb62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e369c52cb54dbfb5704e70e2c47b00, user_id=9902a690b63d48989c8325ea0b80ae3c, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:27.058602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e013d5-2503-4646-a6f3-66456c3deb62 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:27.077144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 973/1939] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:26 2022] PUT /identity/v3/projects/16e369c52cb54dbfb5704e70e2c47b00/users/9902a690b63d48989c8325ea0b80ae3c/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:27.095715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b2219-fb93-4d6d-bc47-65dd94ded17e None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:27.096576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b2219-fb93-4d6d-bc47-65dd94ded17e None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:27.096888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b2219-fb93-4d6d-bc47-65dd94ded17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:27.129409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 967/1940] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:27.130243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-1f6b2219-fb93-4d6d-bc47-65dd94ded17e None None] MFA Rules not processed for user `9902a690b63d48989c8325ea0b80ae3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:27.131903 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1f6b2219-fb93-4d6d-bc47-65dd94ded17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:27.142046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02087b71-d17a-4b72-b7b6-6977926e1699 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:27.145233 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-02087b71-d17a-4b72-b7b6-6977926e1699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:27.250870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 974/1941] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:27.251820 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b2219-fb93-4d6d-bc47-65dd94ded17e None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:27.277327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02087b71-d17a-4b72-b7b6-6977926e1699 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:27.345614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b2219-fb93-4d6d-bc47-65dd94ded17e None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:27.351577 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d428a62-0e11-4cbb-bd60-4341769ab02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:27.352335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d428a62-0e11-4cbb-bd60-4341769ab02c admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:27.352622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d428a62-0e11-4cbb-bd60-4341769ab02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:27.352978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d428a62-0e11-4cbb-bd60-4341769ab02c admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:27.353588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d428a62-0e11-4cbb-bd60-4341769ab02c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:27.356390 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d428a62-0e11-4cbb-bd60-4341769ab02c admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:27.357505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad705147-34f7-4fb7-8f70-98a4887e4a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:27.358863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad705147-34f7-4fb7-8f70-98a4887e4a7b service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:27.359499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad705147-34f7-4fb7-8f70-98a4887e4a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:27.359936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad705147-34f7-4fb7-8f70-98a4887e4a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:27.422006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 975/1942] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:27 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:27.426872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d428a62-0e11-4cbb-bd60-4341769ab02c admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:27.464665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad705147-34f7-4fb7-8f70-98a4887e4a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:27.466131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad705147-34f7-4fb7-8f70-98a4887e4a7b service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:27.491036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:27.494191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 968/1943] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:27.495754 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad705147-34f7-4fb7-8f70-98a4887e4a7b service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:27.497287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ad705147-34f7-4fb7-8f70-98a4887e4a7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:27.524328 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad705147-34f7-4fb7-8f70-98a4887e4a7b service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:27.574654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d428a62-0e11-4cbb-bd60-4341769ab02c admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:27.578956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:27.585988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d97e3c-cc1d-4391-a2f1-c5194b0b3bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:27.587224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d97e3c-cc1d-4391-a2f1-c5194b0b3bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:27.587787 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d97e3c-cc1d-4391-a2f1-c5194b0b3bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:27.588314 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d97e3c-cc1d-4391-a2f1-c5194b0b3bb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:27.589160 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d97e3c-cc1d-4391-a2f1-c5194b0b3bb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:27.598336 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d97e3c-cc1d-4391-a2f1-c5194b0b3bb3 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:27.624217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 976/1944] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:27 2022] POST /identity/v3/users => generated 294 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:27.624217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d97e3c-cc1d-4391-a2f1-c5194b0b3bb3 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:27.625324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:27.627395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:27.627688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:27.627976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] PATH_INFO: `/v3/projects/bfb6cee2478d4091b8ca22c9f47c5ab9` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:27.629931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfb6cee2478d4091b8ca22c9f47c5ab9)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:27.633810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:27.692851 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d97e3c-cc1d-4391-a2f1-c5194b0b3bb3 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:27.699406 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfb6cee2478d4091b8ca22c9f47c5ab9', 'request_id': 'req-cb32a787-f60d-43d8-a0bb-d98ed18719ef'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:27.719682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfb6cee2478d4091b8ca22c9f47c5ab9', 'request_id': 'req-cb32a787-f60d-43d8-a0bb-d98ed18719ef'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:27.729779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfb6cee2478d4091b8ca22c9f47c5ab9', 'request_id': 'req-cb32a787-f60d-43d8-a0bb-d98ed18719ef'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:27.750786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] The token cache is being invalidate because project bfb6cee2478d4091b8ca22c9f47c5ab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:27.751084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:27.752331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:27.755491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 969/1945] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:27 2022] DELETE /identity/v3/projects/bfb6cee2478d4091b8ca22c9f47c5ab9 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:27.757298 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:27.761154 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:27.770222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e85f2-2237-4b0c-a2ce-71541b938993 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:27.771085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e85f2-2237-4b0c-a2ce-71541b938993 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:27.771491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e85f2-2237-4b0c-a2ce-71541b938993 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:27.771867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e85f2-2237-4b0c-a2ce-71541b938993 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:27.774624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-bc1e85f2-2237-4b0c-a2ce-71541b938993 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:27.871774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32a787-f60d-43d8-a0bb-d98ed18719ef admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:27.875422 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e85f2-2237-4b0c-a2ce-71541b938993 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:27.877621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e85f2-2237-4b0c-a2ce-71541b938993 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:27.877902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad311ff-15f9-44ed-8a9e-dd6e8c9a9ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:27.878688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad311ff-15f9-44ed-8a9e-dd6e8c9a9ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:27.879074 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad311ff-15f9-44ed-8a9e-dd6e8c9a9ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:27.879451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad311ff-15f9-44ed-8a9e-dd6e8c9a9ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:27.880314 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad311ff-15f9-44ed-8a9e-dd6e8c9a9ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:27.882083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad311ff-15f9-44ed-8a9e-dd6e8c9a9ef8 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:27.891552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 977/1946] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:27.893175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f2c43-0e03-43f9-a140-50aa1d0bdff9 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:27.893637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f2c43-0e03-43f9-a140-50aa1d0bdff9 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:27.894080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f2c43-0e03-43f9-a140-50aa1d0bdff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:27.898090 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 970/1947] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:27.899742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad311ff-15f9-44ed-8a9e-dd6e8c9a9ef8 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:27.916578 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad311ff-15f9-44ed-8a9e-dd6e8c9a9ef8 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:27.927007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-085f2c43-0e03-43f9-a140-50aa1d0bdff9 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:27.930376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-085f2c43-0e03-43f9-a140-50aa1d0bdff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:27.931944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa5442-c528-4bd9-8244-25e3eea940e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:27.933416 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa5442-c528-4bd9-8244-25e3eea940e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:27.933918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa5442-c528-4bd9-8244-25e3eea940e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:27.934680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa5442-c528-4bd9-8244-25e3eea940e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:27.935927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa5442-c528-4bd9-8244-25e3eea940e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:27.938885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa5442-c528-4bd9-8244-25e3eea940e4 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:27.964530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 978/1948] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:27.968244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7cca7-20dc-401b-9168-aae1360d0c5d None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:27.968244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7cca7-20dc-401b-9168-aae1360d0c5d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:27.968244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7cca7-20dc-401b-9168-aae1360d0c5d None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:27.968244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 979/1949] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:20:27.981487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeadf1-6277-45e6-ab9f-93cf68403363 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:27.982187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeadf1-6277-45e6-ab9f-93cf68403363 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:27.983102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeadf1-6277-45e6-ab9f-93cf68403363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:27.986998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 971/1950] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:27.991264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa5442-c528-4bd9-8244-25e3eea940e4 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:28.009436 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa5442-c528-4bd9-8244-25e3eea940e4 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:28.012228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-cdeeadf1-6277-45e6-ab9f-93cf68403363 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:28.013626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cdeeadf1-6277-45e6-ab9f-93cf68403363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:28.021428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce12aa-88e0-4096-b786-d83c17eeb0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:28.023145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce12aa-88e0-4096-b786-d83c17eeb0a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:28.023583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce12aa-88e0-4096-b786-d83c17eeb0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:28.024440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce12aa-88e0-4096-b786-d83c17eeb0a8 admin admin] PATH_INFO: `/v3/projects/d5f0d1d0426a4ec1b88f835c132b1f97/users/61cd93a17e8246ad88caec87616ad594/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:28.033353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 980/1951] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:28.034503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86390366-0562-4c90-aa07-c048e3f5c9e0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:28.034922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86390366-0562-4c90-aa07-c048e3f5c9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:28.035455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86390366-0562-4c90-aa07-c048e3f5c9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:28.047098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:28.052562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce12aa-88e0-4096-b786-d83c17eeb0a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5f0d1d0426a4ec1b88f835c132b1f97, user_id=61cd93a17e8246ad88caec87616ad594, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:28.063233 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce12aa-88e0-4096-b786-d83c17eeb0a8 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:28.089739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-86390366-0562-4c90-aa07-c048e3f5c9e0 None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:28.094244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-86390366-0562-4c90-aa07-c048e3f5c9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:28.111665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 972/1952] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:27 2022] PUT /identity/v3/projects/d5f0d1d0426a4ec1b88f835c132b1f97/users/61cd93a17e8246ad88caec87616ad594/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:28.111665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce12aa-88e0-4096-b786-d83c17eeb0a8 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:28.161742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:28.258698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 981/1953] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:20:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:28.265209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4376ba5-1f17-4147-ab64-11f1d50f8a48 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:28.266226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4376ba5-1f17-4147-ab64-11f1d50f8a48 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:28.266615 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4376ba5-1f17-4147-ab64-11f1d50f8a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:28.317355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce12aa-88e0-4096-b786-d83c17eeb0a8 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:28.334900 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626095de-39db-4636-bf62-66455ca76001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:28.335599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626095de-39db-4636-bf62-66455ca76001 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:28.335958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626095de-39db-4636-bf62-66455ca76001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:28.336171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626095de-39db-4636-bf62-66455ca76001 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:28.338740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626095de-39db-4636-bf62-66455ca76001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:28.339341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626095de-39db-4636-bf62-66455ca76001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:28.341077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626095de-39db-4636-bf62-66455ca76001 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:28.344308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-b4376ba5-1f17-4147-ab64-11f1d50f8a48 None None] MFA Rules not processed for user `61cd93a17e8246ad88caec87616ad594`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:28.345639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b4376ba5-1f17-4147-ab64-11f1d50f8a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:28.367551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 973/1954] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:20:28.375016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626095de-39db-4636-bf62-66455ca76001 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:28.419211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626095de-39db-4636-bf62-66455ca76001 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:28.428874 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb7352-7649-490a-a89e-4eb2adfbc44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:28.430562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb7352-7649-490a-a89e-4eb2adfbc44d admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:28.430810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb7352-7649-490a-a89e-4eb2adfbc44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:28.431553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb7352-7649-490a-a89e-4eb2adfbc44d admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:28.431780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb7352-7649-490a-a89e-4eb2adfbc44d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:28.438187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb7352-7649-490a-a89e-4eb2adfbc44d admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:28.518267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 974/1955] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:28 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:20:28.519270 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb7352-7649-490a-a89e-4eb2adfbc44d admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:28.522378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-8deb7352-7649-490a-a89e-4eb2adfbc44d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:28.530191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 982/1956] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:28.530191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4376ba5-1f17-4147-ab64-11f1d50f8a48 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:28.530563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b4376ba5-1f17-4147-ab64-11f1d50f8a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:28.552586 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4376ba5-1f17-4147-ab64-11f1d50f8a48 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:28.595825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb7352-7649-490a-a89e-4eb2adfbc44d admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:28.601621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:28.602665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:28.603059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:28.603443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] PATH_INFO: `/v3/users/1fdb7235f8164c49b9559ceb491ba47d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:28.605053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fdb7235f8164c49b9559ceb491ba47d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:28.607113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:28.646944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ad7b3-4e7d-408c-8e8e-b33d8ca376c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:28.647631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ad7b3-4e7d-408c-8e8e-b33d8ca376c8 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:28.647920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ad7b3-4e7d-408c-8e8e-b33d8ca376c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:28.648230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ad7b3-4e7d-408c-8e8e-b33d8ca376c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:28.659525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ad7b3-4e7d-408c-8e8e-b33d8ca376c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:28.661649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ad7b3-4e7d-408c-8e8e-b33d8ca376c8 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:28.673008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fdb7235f8164c49b9559ceb491ba47d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:28.673807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fdb7235f8164c49b9559ceb491ba47d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:28.676275 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 983/1957] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:28.678296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ad7b3-4e7d-408c-8e8e-b33d8ca376c8 service nova] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:28.687608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fdb7235f8164c49b9559ceb491ba47d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:28.695143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ad7b3-4e7d-408c-8e8e-b33d8ca376c8 service nova] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:28.703425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf836c-ef1b-4b55-a6c9-8429ab4b985d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:28.704277 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf836c-ef1b-4b55-a6c9-8429ab4b985d admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:28.704830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf836c-ef1b-4b55-a6c9-8429ab4b985d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:28.705214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf836c-ef1b-4b55-a6c9-8429ab4b985d admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:28.706323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf836c-ef1b-4b55-a6c9-8429ab4b985d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:28.708792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf836c-ef1b-4b55-a6c9-8429ab4b985d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:28.712723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fdb7235f8164c49b9559ceb491ba47d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:28.713597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fdb7235f8164c49b9559ceb491ba47d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:28.727912 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:28.743497 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 975/1958] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:28 2022] DELETE /identity/v3/users/1fdb7235f8164c49b9559ceb491ba47d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:28.743497 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:28.744187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:28.759480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 984/1959] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:28 2022] POST /identity/v3/users => generated 292 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:28.765626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ff57d-a310-4389-b428-eeccef45606e None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:28.766045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ff57d-a310-4389-b428-eeccef45606e None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:28.766421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ff57d-a310-4389-b428-eeccef45606e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:28.770797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:28.801363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-203ff57d-a310-4389-b428-eeccef45606e None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:28.802727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-203ff57d-a310-4389-b428-eeccef45606e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:28.851748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448cd3e-2fcf-4a24-8934-db51dcf16c17 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:28.863816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b5804-2db1-4233-8649-b955058b7ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:28.864742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b5804-2db1-4233-8649-b955058b7ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:28.865833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b5804-2db1-4233-8649-b955058b7ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:28.866269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b5804-2db1-4233-8649-b955058b7ef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:28.867140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b5804-2db1-4233-8649-b955058b7ef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:28.869339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b5804-2db1-4233-8649-b955058b7ef8 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:28.918459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 976/1960] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:28 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:28.918459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b5804-2db1-4233-8649-b955058b7ef8 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:28.921225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-322b5804-2db1-4233-8649-b955058b7ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:28.955734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 985/1961] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:28.956494 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ff57d-a310-4389-b428-eeccef45606e None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:28.984975 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ff57d-a310-4389-b428-eeccef45606e None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:29.020176 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b5804-2db1-4233-8649-b955058b7ef8 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:29.029390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aba3b-b2aa-4477-857e-c519c9d99009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:29.029390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aba3b-b2aa-4477-857e-c519c9d99009 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.029390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aba3b-b2aa-4477-857e-c519c9d99009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.029690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aba3b-b2aa-4477-857e-c519c9d99009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.034328 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2aba3b-b2aa-4477-857e-c519c9d99009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:29.034328 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2aba3b-b2aa-4477-857e-c519c9d99009 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:29.050996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 977/1962] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:29.051510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aba3b-b2aa-4477-857e-c519c9d99009 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:29.053505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ea2aba3b-b2aa-4477-857e-c519c9d99009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:29.093833 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e3251-fff5-49cf-bdd2-086e01eecb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:29.094992 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e3251-fff5-49cf-bdd2-086e01eecb9e admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.095581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e3251-fff5-49cf-bdd2-086e01eecb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.095989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e3251-fff5-49cf-bdd2-086e01eecb9e admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.096733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e3251-fff5-49cf-bdd2-086e01eecb9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:29.098870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e3251-fff5-49cf-bdd2-086e01eecb9e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:29.135367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 986/1963] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:28 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:29.136994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e3251-fff5-49cf-bdd2-086e01eecb9e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:29.155310 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aba3b-b2aa-4477-857e-c519c9d99009 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:29.159129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e3251-fff5-49cf-bdd2-086e01eecb9e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:29.163265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b51b0-f7b1-4149-b736-d84660558d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:29.163987 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b51b0-f7b1-4149-b736-d84660558d20 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.164421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b51b0-f7b1-4149-b736-d84660558d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.164983 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b51b0-f7b1-4149-b736-d84660558d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.171453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb366c4-5e98-44c3-b6db-e28556820834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:29.172393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb366c4-5e98-44c3-b6db-e28556820834 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.178264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb366c4-5e98-44c3-b6db-e28556820834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.178264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb366c4-5e98-44c3-b6db-e28556820834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.178264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb366c4-5e98-44c3-b6db-e28556820834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:29.178264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb366c4-5e98-44c3-b6db-e28556820834 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:29.190039 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b51b0-f7b1-4149-b736-d84660558d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:29.191213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 987/1964] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:29.192900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb366c4-5e98-44c3-b6db-e28556820834 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:29.193691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b51b0-f7b1-4149-b736-d84660558d20 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:29.204598 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb366c4-5e98-44c3-b6db-e28556820834 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:29.210768 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202037c3-a4f5-4b61-8afc-834ec6813374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:29.212329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202037c3-a4f5-4b61-8afc-834ec6813374 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.212711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202037c3-a4f5-4b61-8afc-834ec6813374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.213085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202037c3-a4f5-4b61-8afc-834ec6813374 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.214006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202037c3-a4f5-4b61-8afc-834ec6813374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:29.215725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202037c3-a4f5-4b61-8afc-834ec6813374 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:29.221455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 978/1965] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:29.234156 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 988/1966] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:29.235837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202037c3-a4f5-4b61-8afc-834ec6813374 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:29.236327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b51b0-f7b1-4149-b736-d84660558d20 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:29.265114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b51b0-f7b1-4149-b736-d84660558d20 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:29.265333 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202037c3-a4f5-4b61-8afc-834ec6813374 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:29.271056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadcba6-8e02-47db-8cb9-6e4bcee62437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:29.272096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadcba6-8e02-47db-8cb9-6e4bcee62437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.272400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadcba6-8e02-47db-8cb9-6e4bcee62437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.272672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadcba6-8e02-47db-8cb9-6e4bcee62437 admin admin] PATH_INFO: `/v3/projects/d6820e7741244585b8dbc9980d7d605f/users/fe22ded872ab4ed893216296cfea091f/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.274357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9ef0a-ccbb-40a0-bded-658a4dc927cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:29.276197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9ef0a-ccbb-40a0-bded-658a4dc927cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.283116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9ef0a-ccbb-40a0-bded-658a4dc927cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.285209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9ef0a-ccbb-40a0-bded-658a4dc927cd admin admin] PATH_INFO: `/v3/projects/cff1ab65510843838f21195007737319/users/9262f80f454a4721beaff65e97f6b55d/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.301340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aadcba6-8e02-47db-8cb9-6e4bcee62437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6820e7741244585b8dbc9980d7d605f, user_id=fe22ded872ab4ed893216296cfea091f, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:29.304217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aadcba6-8e02-47db-8cb9-6e4bcee62437 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:29.307292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9ef0a-ccbb-40a0-bded-658a4dc927cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff1ab65510843838f21195007737319, user_id=9262f80f454a4721beaff65e97f6b55d, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:29.310853 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9ef0a-ccbb-40a0-bded-658a4dc927cd admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:29.320569 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 989/1967] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:29 2022] PUT /identity/v3/projects/d6820e7741244585b8dbc9980d7d605f/users/fe22ded872ab4ed893216296cfea091f/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:29.324890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadcba6-8e02-47db-8cb9-6e4bcee62437 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:29.329709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 979/1968] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:29 2022] PUT /identity/v3/projects/cff1ab65510843838f21195007737319/users/9262f80f454a4721beaff65e97f6b55d/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:29.341439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf2a09-13b5-40dc-bbb2-e14fb742e6fc None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.342029 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf2a09-13b5-40dc-bbb2-e14fb742e6fc None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.342489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf2a09-13b5-40dc-bbb2-e14fb742e6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.369601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:29.372886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-fbaf2a09-13b5-40dc-bbb2-e14fb742e6fc None None] MFA Rules not processed for user `9262f80f454a4721beaff65e97f6b55d`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:29.374554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-fbaf2a09-13b5-40dc-bbb2-e14fb742e6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:29.378873 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:29.427019 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadcba6-8e02-47db-8cb9-6e4bcee62437 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:29.463184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 980/1969] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:29.468159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf2a09-13b5-40dc-bbb2-e14fb742e6fc None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:29.504431 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a75992-378d-448b-a8d3-d5e660ed6593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:29.505114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a75992-378d-448b-a8d3-d5e660ed6593 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.505397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a75992-378d-448b-a8d3-d5e660ed6593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.505676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a75992-378d-448b-a8d3-d5e660ed6593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.506517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a75992-378d-448b-a8d3-d5e660ed6593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:29.509522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a75992-378d-448b-a8d3-d5e660ed6593 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:29.528240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 990/1970] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:29.531663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a75992-378d-448b-a8d3-d5e660ed6593 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:29.544526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a75992-378d-448b-a8d3-d5e660ed6593 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:29.551415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf2a09-13b5-40dc-bbb2-e14fb742e6fc None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:29.558584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e198e-48d6-4261-9607-60294e6f263b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:29.559908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e198e-48d6-4261-9607-60294e6f263b admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.560416 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e198e-48d6-4261-9607-60294e6f263b admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.560740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e198e-48d6-4261-9607-60294e6f263b admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.563188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0535d-d328-4554-b7db-0dc9e576999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:29.564091 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e198e-48d6-4261-9607-60294e6f263b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:29.564248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0535d-d328-4554-b7db-0dc9e576999d admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.564538 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0535d-d328-4554-b7db-0dc9e576999d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.565517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0535d-d328-4554-b7db-0dc9e576999d admin admin] PATH_INFO: `/v3/projects/d6820e7741244585b8dbc9980d7d605f/users/fe22ded872ab4ed893216296cfea091f/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.566267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e198e-48d6-4261-9607-60294e6f263b admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:29.567723 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0535d-d328-4554-b7db-0dc9e576999d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6820e7741244585b8dbc9980d7d605f, user_id=fe22ded872ab4ed893216296cfea091f, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:29.570641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0535d-d328-4554-b7db-0dc9e576999d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:29.585883 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 991/1971] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:29 2022] PUT /identity/v3/projects/d6820e7741244585b8dbc9980d7d605f/users/fe22ded872ab4ed893216296cfea091f/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:29.588901 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0535d-d328-4554-b7db-0dc9e576999d admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:29.590686 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ada0535d-d328-4554-b7db-0dc9e576999d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:29.602107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 981/1972] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:29 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:20:29.605710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0d9b7-ce11-4809-9b56-3fa5615c62ed None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.606015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0d9b7-ce11-4809-9b56-3fa5615c62ed None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.606494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0d9b7-ce11-4809-9b56-3fa5615c62ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.658569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-9fb0d9b7-ce11-4809-9b56-3fa5615c62ed None None] MFA Rules not processed for user `fe22ded872ab4ed893216296cfea091f`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:29.660001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9fb0d9b7-ce11-4809-9b56-3fa5615c62ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:29.707331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0535d-d328-4554-b7db-0dc9e576999d admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:29.773154 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 982/1973] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:29.774864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0d9b7-ce11-4809-9b56-3fa5615c62ed None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:29.812901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:29.822138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:29.822580 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.822935 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.823220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] PATH_INFO: `/v3/projects/082383cd98734e94bc8ba7275ea2fdff` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.824637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=082383cd98734e94bc8ba7275ea2fdff)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:29.826583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:29.848264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:29.877166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0d9b7-ce11-4809-9b56-3fa5615c62ed None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:29.889726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '082383cd98734e94bc8ba7275ea2fdff', 'request_id': 'req-702a03b5-8c94-439a-81ed-d53cc3ddfc92'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:29.896419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '082383cd98734e94bc8ba7275ea2fdff', 'request_id': 'req-702a03b5-8c94-439a-81ed-d53cc3ddfc92'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:29.903030 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '082383cd98734e94bc8ba7275ea2fdff', 'request_id': 'req-702a03b5-8c94-439a-81ed-d53cc3ddfc92'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:29.924824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] The token cache is being invalidate because project 082383cd98734e94bc8ba7275ea2fdff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:29.925427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:29.926396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:29.928939 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 992/1974] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:29 2022] DELETE /identity/v3/projects/082383cd98734e94bc8ba7275ea2fdff => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:29.931932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:29.931932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:29.954371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6ece4-bd26-4cf0-87f7-84530552e9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:29.955131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6ece4-bd26-4cf0-87f7-84530552e9ae admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:29.955339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6ece4-bd26-4cf0-87f7-84530552e9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:29.955623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6ece4-bd26-4cf0-87f7-84530552e9ae admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:29.956363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6ece4-bd26-4cf0-87f7-84530552e9ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:29.958444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6ece4-bd26-4cf0-87f7-84530552e9ae admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:29.988491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 983/1975] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:29 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:29.988491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6ece4-bd26-4cf0-87f7-84530552e9ae admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:29.989665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b4e6ece4-bd26-4cf0-87f7-84530552e9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:30.005391 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6ece4-bd26-4cf0-87f7-84530552e9ae admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:30.005640 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a03b5-8c94-439a-81ed-d53cc3ddfc92 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:30.009770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86592409-83fe-46ec-992f-ecabe1d6a044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:30.010609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86592409-83fe-46ec-992f-ecabe1d6a044 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:30.011127 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86592409-83fe-46ec-992f-ecabe1d6a044 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:30.011564 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86592409-83fe-46ec-992f-ecabe1d6a044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:30.011683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:30.012939 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:30.013210 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:30.013493 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] PATH_INFO: `/v3/users/ac953a40a80744299eb4a1fff7424da2` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:30.014251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-86592409-83fe-46ec-992f-ecabe1d6a044 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:30.014850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac953a40a80744299eb4a1fff7424da2)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:30.017216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:30.118802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86592409-83fe-46ec-992f-ecabe1d6a044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:30.121207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86592409-83fe-46ec-992f-ecabe1d6a044 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:30.121901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac953a40a80744299eb4a1fff7424da2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:30.124834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac953a40a80744299eb4a1fff7424da2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:30.135237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 993/1976] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:30.136142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86592409-83fe-46ec-992f-ecabe1d6a044 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:30.137559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-86592409-83fe-46ec-992f-ecabe1d6a044 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:30.141461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac953a40a80744299eb4a1fff7424da2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:30.152931 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86592409-83fe-46ec-992f-ecabe1d6a044 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:30.157084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac953a40a80744299eb4a1fff7424da2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:30.157892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac953a40a80744299eb4a1fff7424da2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:30.158263 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765b98b-eaf1-4af9-a981-e6681228f12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:30.158964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765b98b-eaf1-4af9-a981-e6681228f12a admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:30.159255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765b98b-eaf1-4af9-a981-e6681228f12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:30.159881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765b98b-eaf1-4af9-a981-e6681228f12a admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:30.160559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765b98b-eaf1-4af9-a981-e6681228f12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:30.162906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765b98b-eaf1-4af9-a981-e6681228f12a admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:30.176102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 994/1977] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:30.178233 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 984/1978] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:29 2022] DELETE /identity/v3/users/ac953a40a80744299eb4a1fff7424da2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:30.182687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765b98b-eaf1-4af9-a981-e6681228f12a admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:30.184481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8765b98b-eaf1-4af9-a981-e6681228f12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:30.198236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765b98b-eaf1-4af9-a981-e6681228f12a admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:30.246470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:30.265653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa19e0-a6f0-4346-a0a8-7ca7d4e580a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:30.266347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa19e0-a6f0-4346-a0a8-7ca7d4e580a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:30.266698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa19e0-a6f0-4346-a0a8-7ca7d4e580a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:30.266949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa19e0-a6f0-4346-a0a8-7ca7d4e580a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:30.267784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa19e0-a6f0-4346-a0a8-7ca7d4e580a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:30.269456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa19e0-a6f0-4346-a0a8-7ca7d4e580a7 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:30.284228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 995/1979] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:30.285642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:30.300498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4965e-8123-440f-850f-4af0667e9b5f admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:30.316343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef95eaa-0ce0-4a78-ad25-5d7adf7acb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:30.317414 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef95eaa-0ce0-4a78-ad25-5d7adf7acb21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:30.317804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef95eaa-0ce0-4a78-ad25-5d7adf7acb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:30.318198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef95eaa-0ce0-4a78-ad25-5d7adf7acb21 admin admin] PATH_INFO: `/v3/projects/3a8c2e4c7ebf4d57998f55034e87fc30/users/246fee748ecd414cbef678442620dd5f/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:30.359147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef95eaa-0ce0-4a78-ad25-5d7adf7acb21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a8c2e4c7ebf4d57998f55034e87fc30, user_id=246fee748ecd414cbef678442620dd5f, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:30.363740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef95eaa-0ce0-4a78-ad25-5d7adf7acb21 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:30.379949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:30.412189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 985/1980] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:30 2022] PUT /identity/v3/projects/3a8c2e4c7ebf4d57998f55034e87fc30/users/246fee748ecd414cbef678442620dd5f/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:30.415550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef3670-6105-47eb-a2bc-420b95b749b7 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:30.416347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef3670-6105-47eb-a2bc-420b95b749b7 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:30.416862 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef3670-6105-47eb-a2bc-420b95b749b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:30.481888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-ccef3670-6105-47eb-a2bc-420b95b749b7 None None] MFA Rules not processed for user `246fee748ecd414cbef678442620dd5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:30.486653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ccef3670-6105-47eb-a2bc-420b95b749b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:30.614541 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 996/1981] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:30.615236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef95eaa-0ce0-4a78-ad25-5d7adf7acb21 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:30.627283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef3670-6105-47eb-a2bc-420b95b749b7 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:30.629984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ccef3670-6105-47eb-a2bc-420b95b749b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:30.738379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef3670-6105-47eb-a2bc-420b95b749b7 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:30.745873 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbabdd-0fe9-464d-bb0d-63d160790880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:30.746978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbabdd-0fe9-464d-bb0d-63d160790880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:30.747487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbabdd-0fe9-464d-bb0d-63d160790880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:30.747875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbabdd-0fe9-464d-bb0d-63d160790880 admin admin] PATH_INFO: `/v3/projects/05545bb5fb46427d9aae6434b651715a` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:30.749375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cbabdd-0fe9-464d-bb0d-63d160790880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05545bb5fb46427d9aae6434b651715a)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:30.751837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cbabdd-0fe9-464d-bb0d-63d160790880 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:30.774600 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef95eaa-0ce0-4a78-ad25-5d7adf7acb21 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:30.786837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d19de2-605d-4111-9ead-25b0fbbee7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:30.787825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d19de2-605d-4111-9ead-25b0fbbee7b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:30.788373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d19de2-605d-4111-9ead-25b0fbbee7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:30.788828 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d19de2-605d-4111-9ead-25b0fbbee7b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:30.791377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d19de2-605d-4111-9ead-25b0fbbee7b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:30.794481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d19de2-605d-4111-9ead-25b0fbbee7b3 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:30.848712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 986/1982] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:30 2022] POST /identity/v3/projects => generated 381 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:30.850031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d19de2-605d-4111-9ead-25b0fbbee7b3 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:30.859411 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-00cbabdd-0fe9-464d-bb0d-63d160790880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05545bb5fb46427d9aae6434b651715a', 'request_id': 'req-00cbabdd-0fe9-464d-bb0d-63d160790880'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:30.876302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-00cbabdd-0fe9-464d-bb0d-63d160790880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05545bb5fb46427d9aae6434b651715a', 'request_id': 'req-00cbabdd-0fe9-464d-bb0d-63d160790880'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:30.892920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-00cbabdd-0fe9-464d-bb0d-63d160790880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05545bb5fb46427d9aae6434b651715a', 'request_id': 'req-00cbabdd-0fe9-464d-bb0d-63d160790880'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:30.919473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:30.926735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-00cbabdd-0fe9-464d-bb0d-63d160790880 admin admin] The token cache is being invalidate because project 05545bb5fb46427d9aae6434b651715a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:30.927174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-00cbabdd-0fe9-464d-bb0d-63d160790880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:30.928178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-00cbabdd-0fe9-464d-bb0d-63d160790880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:30.931092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 997/1983] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:30 2022] DELETE /identity/v3/projects/05545bb5fb46427d9aae6434b651715a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:30.935946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be10ae-64dd-4741-8bbb-a042480de787 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:30.936273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be10ae-64dd-4741-8bbb-a042480de787 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:30.936702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be10ae-64dd-4741-8bbb-a042480de787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:30.970487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-19be10ae-64dd-4741-8bbb-a042480de787 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:30.971859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-19be10ae-64dd-4741-8bbb-a042480de787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:31.009357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d19de2-605d-4111-9ead-25b0fbbee7b3 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.024691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c48ed-6c7e-4c12-baee-0003248b40a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.025355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c48ed-6c7e-4c12-baee-0003248b40a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.026578 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c48ed-6c7e-4c12-baee-0003248b40a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.027898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c48ed-6c7e-4c12-baee-0003248b40a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.029330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c48ed-6c7e-4c12-baee-0003248b40a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:31.033544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c48ed-6c7e-4c12-baee-0003248b40a5 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:31.088739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 987/1984] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:30 2022] POST /identity/v3/users => generated 297 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:31.092583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c48ed-6c7e-4c12-baee-0003248b40a5 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:31.094501 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d50c48ed-6c7e-4c12-baee-0003248b40a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:31.119708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c48ed-6c7e-4c12-baee-0003248b40a5 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.125976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c58a1a-dfe1-410d-a000-37ae492ac99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.126656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c58a1a-dfe1-410d-a000-37ae492ac99e admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.126944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c58a1a-dfe1-410d-a000-37ae492ac99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.127223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c58a1a-dfe1-410d-a000-37ae492ac99e admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.129347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c58a1a-dfe1-410d-a000-37ae492ac99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:31.132134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 998/1985] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:31.141296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c58a1a-dfe1-410d-a000-37ae492ac99e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:31.151714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfdc84-e0c2-4216-b4fb-9b696b377d7b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.152237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfdc84-e0c2-4216-b4fb-9b696b377d7b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.152659 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfdc84-e0c2-4216-b4fb-9b696b377d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.154483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 988/1986] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:31.158704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c58a1a-dfe1-410d-a000-37ae492ac99e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:31.181772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c58a1a-dfe1-410d-a000-37ae492ac99e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.187054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fea19-8c9e-4b76-b5d9-234ceeaead84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.188294 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fea19-8c9e-4b76-b5d9-234ceeaead84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.188583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fea19-8c9e-4b76-b5d9-234ceeaead84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.188867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fea19-8c9e-4b76-b5d9-234ceeaead84 admin admin] PATH_INFO: `/v3/projects/e3811a42b04e48889fc7577cdff28816/users/d14fbc32b79e41d1ba01b29141d93865/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.194809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-dabfdc84-e0c2-4216-b4fb-9b696b377d7b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:31.197303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-dabfdc84-e0c2-4216-b4fb-9b696b377d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:31.216198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 999/1987] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:31.216198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfdc84-e0c2-4216-b4fb-9b696b377d7b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:31.218183 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-dabfdc84-e0c2-4216-b4fb-9b696b377d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:31.223450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fea19-8c9e-4b76-b5d9-234ceeaead84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3811a42b04e48889fc7577cdff28816, user_id=d14fbc32b79e41d1ba01b29141d93865, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:31.226488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fea19-8c9e-4b76-b5d9-234ceeaead84 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:31.248487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 989/1988] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:31 2022] PUT /identity/v3/projects/e3811a42b04e48889fc7577cdff28816/users/d14fbc32b79e41d1ba01b29141d93865/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:31.249478 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fea19-8c9e-4b76-b5d9-234ceeaead84 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:31.307721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfdc84-e0c2-4216-b4fb-9b696b377d7b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.318461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:31.353146 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fea19-8c9e-4b76-b5d9-234ceeaead84 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.358691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f3fb-7185-4721-b33e-f1118c2cdf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.359394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f3fb-7185-4721-b33e-f1118c2cdf79 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.359683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f3fb-7185-4721-b33e-f1118c2cdf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.359963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f3fb-7185-4721-b33e-f1118c2cdf79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.360788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f3fb-7185-4721-b33e-f1118c2cdf79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:31.361212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f3fb-7185-4721-b33e-f1118c2cdf79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:31.363529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f3fb-7185-4721-b33e-f1118c2cdf79 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:31.399166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 990/1989] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:20:31.399839 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f3fb-7185-4721-b33e-f1118c2cdf79 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:31.413833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f3fb-7185-4721-b33e-f1118c2cdf79 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.421031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96adb7-863a-41d1-8525-80193b82f1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.421456 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96adb7-863a-41d1-8525-80193b82f1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.421871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96adb7-863a-41d1-8525-80193b82f1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.422250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96adb7-863a-41d1-8525-80193b82f1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.423098 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96adb7-863a-41d1-8525-80193b82f1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:31.424881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96adb7-863a-41d1-8525-80193b82f1a7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:31.441127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 991/1990] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:31.441127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96adb7-863a-41d1-8525-80193b82f1a7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:31.444915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ac5c9-63af-440d-b42e-9fa6d7c18be4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.447144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ac5c9-63af-440d-b42e-9fa6d7c18be4 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.447475 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ac5c9-63af-440d-b42e-9fa6d7c18be4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.447762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ac5c9-63af-440d-b42e-9fa6d7c18be4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.453523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-911ac5c9-63af-440d-b42e-9fa6d7c18be4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:31.463157 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96adb7-863a-41d1-8525-80193b82f1a7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.468762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4b4b2-e6e2-4fdd-a78d-c0b57663f0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.469831 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4b4b2-e6e2-4fdd-a78d-c0b57663f0e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.470202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4b4b2-e6e2-4fdd-a78d-c0b57663f0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.470734 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4b4b2-e6e2-4fdd-a78d-c0b57663f0e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.471510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4b4b2-e6e2-4fdd-a78d-c0b57663f0e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:31.474470 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4b4b2-e6e2-4fdd-a78d-c0b57663f0e7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:31.517081 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 992/1991] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:31 2022] POST /identity/v3/projects => generated 375 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:31.517081 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4b4b2-e6e2-4fdd-a78d-c0b57663f0e7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:31.550026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ac5c9-63af-440d-b42e-9fa6d7c18be4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:31.551344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ac5c9-63af-440d-b42e-9fa6d7c18be4 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:31.563638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1000/1992] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:31.564166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ac5c9-63af-440d-b42e-9fa6d7c18be4 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:31.580147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:31.580147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ac5c9-63af-440d-b42e-9fa6d7c18be4 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.642760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4b4b2-e6e2-4fdd-a78d-c0b57663f0e7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.656801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e5ca5-1ade-4f28-88f4-3d854158ff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.658299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e5ca5-1ade-4f28-88f4-3d854158ff21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.658884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e5ca5-1ade-4f28-88f4-3d854158ff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.659868 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e5ca5-1ade-4f28-88f4-3d854158ff21 admin admin] PATH_INFO: `/v3/projects/e3811a42b04e48889fc7577cdff28816/users/d14fbc32b79e41d1ba01b29141d93865/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.663402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e5ca5-1ade-4f28-88f4-3d854158ff21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3811a42b04e48889fc7577cdff28816, user_id=d14fbc32b79e41d1ba01b29141d93865, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:31.668639 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e5ca5-1ade-4f28-88f4-3d854158ff21 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:31.677377 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45a259-d47e-472e-ae0d-4089954cfafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.678063 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45a259-d47e-472e-ae0d-4089954cfafe admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.678350 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45a259-d47e-472e-ae0d-4089954cfafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.678730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45a259-d47e-472e-ae0d-4089954cfafe admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.679426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45a259-d47e-472e-ae0d-4089954cfafe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:31.681574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45a259-d47e-472e-ae0d-4089954cfafe admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:31.684099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 993/1993] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:31 2022] PUT /identity/v3/projects/e3811a42b04e48889fc7577cdff28816/users/d14fbc32b79e41d1ba01b29141d93865/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:31.692233 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bc7f8-2734-4525-b478-574fad610702 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.692704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bc7f8-2734-4525-b478-574fad610702 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.693049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bc7f8-2734-4525-b478-574fad610702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.703218 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1001/1994] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:31 2022] POST /identity/v3/users => generated 288 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:31.707119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45a259-d47e-472e-ae0d-4089954cfafe admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:31.720222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45a259-d47e-472e-ae0d-4089954cfafe admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.722275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-975bc7f8-2734-4525-b478-574fad610702 None None] MFA Rules not processed for user `d14fbc32b79e41d1ba01b29141d93865`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:31.723445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-975bc7f8-2734-4525-b478-574fad610702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:31.795521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046efb8c-3d84-4df5-b5b9-1a9bd5a66b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.796332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046efb8c-3d84-4df5-b5b9-1a9bd5a66b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.796591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046efb8c-3d84-4df5-b5b9-1a9bd5a66b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.796875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046efb8c-3d84-4df5-b5b9-1a9bd5a66b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.797765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046efb8c-3d84-4df5-b5b9-1a9bd5a66b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:31.799401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046efb8c-3d84-4df5-b5b9-1a9bd5a66b2b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:31.810478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1002/1995] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:31.814658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046efb8c-3d84-4df5-b5b9-1a9bd5a66b2b admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:31.825574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046efb8c-3d84-4df5-b5b9-1a9bd5a66b2b admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.830829 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e0b31-6f13-4803-9b3e-db7b2bb72caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.831592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e0b31-6f13-4803-9b3e-db7b2bb72caf admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.831887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e0b31-6f13-4803-9b3e-db7b2bb72caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.832208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e0b31-6f13-4803-9b3e-db7b2bb72caf admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.832944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e0b31-6f13-4803-9b3e-db7b2bb72caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:31.834281 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 994/1996] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:20:31.839603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e0b31-6f13-4803-9b3e-db7b2bb72caf admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:31.848036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bc7f8-2734-4525-b478-574fad610702 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:31.849988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-975bc7f8-2734-4525-b478-574fad610702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:31.866257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1003/1997] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:31.867937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e0b31-6f13-4803-9b3e-db7b2bb72caf admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:31.875972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:31.890235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e0b31-6f13-4803-9b3e-db7b2bb72caf admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.904756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927470a2-9e10-4cc9-9ef3-e0294c61bb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.905683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927470a2-9e10-4cc9-9ef3-e0294c61bb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.905976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927470a2-9e10-4cc9-9ef3-e0294c61bb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.906258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927470a2-9e10-4cc9-9ef3-e0294c61bb86 admin admin] PATH_INFO: `/v3/projects/f82081e918504de7b3505338d7d8c22f/users/357ece69b4ec4b1383cd75b205c25879/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.924922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927470a2-9e10-4cc9-9ef3-e0294c61bb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f82081e918504de7b3505338d7d8c22f, user_id=357ece69b4ec4b1383cd75b205c25879, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:31.927840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927470a2-9e10-4cc9-9ef3-e0294c61bb86 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:31.938134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bc7f8-2734-4525-b478-574fad610702 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:31.938134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d38d8-a496-4502-ad4b-9b9d83221853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:31.938659 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d38d8-a496-4502-ad4b-9b9d83221853 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.938838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d38d8-a496-4502-ad4b-9b9d83221853 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.939104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d38d8-a496-4502-ad4b-9b9d83221853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:31.941307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-1f5d38d8-a496-4502-ad4b-9b9d83221853 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:31.956643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1004/1998] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:31 2022] PUT /identity/v3/projects/f82081e918504de7b3505338d7d8c22f/users/357ece69b4ec4b1383cd75b205c25879/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:31.969099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131294d-3fcd-42ba-b5e1-0ee9d62cdbe8 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:31.969524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131294d-3fcd-42ba-b5e1-0ee9d62cdbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:31.969911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131294d-3fcd-42ba-b5e1-0ee9d62cdbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:32.005338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-b131294d-3fcd-42ba-b5e1-0ee9d62cdbe8 None None] MFA Rules not processed for user `357ece69b4ec4b1383cd75b205c25879`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:32.007180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b131294d-3fcd-42ba-b5e1-0ee9d62cdbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:32.029496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d38d8-a496-4502-ad4b-9b9d83221853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:32.031035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d38d8-a496-4502-ad4b-9b9d83221853 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:32.042353 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 995/1999] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:32.064676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:32.089571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1005/2000] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:32.094125 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d38d8-a496-4502-ad4b-9b9d83221853 service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:32.106271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d38d8-a496-4502-ad4b-9b9d83221853 service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:32.208857 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f91220-f149-479c-8f1f-b764b98f85e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:32.209650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f91220-f149-479c-8f1f-b764b98f85e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:32.209955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f91220-f149-479c-8f1f-b764b98f85e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:32.210289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f91220-f149-479c-8f1f-b764b98f85e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:32.211049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f91220-f149-479c-8f1f-b764b98f85e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:32.214074 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f91220-f149-479c-8f1f-b764b98f85e2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:32.268914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 996/2001] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:32 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:32.272926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131294d-3fcd-42ba-b5e1-0ee9d62cdbe8 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:32.290782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131294d-3fcd-42ba-b5e1-0ee9d62cdbe8 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:32.362489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6703b-6e0e-4000-9fbe-d5363a9ada93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:32.363380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6703b-6e0e-4000-9fbe-d5363a9ada93 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:32.363672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6703b-6e0e-4000-9fbe-d5363a9ada93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:32.363958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6703b-6e0e-4000-9fbe-d5363a9ada93 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:32.364724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6703b-6e0e-4000-9fbe-d5363a9ada93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:32.367037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6703b-6e0e-4000-9fbe-d5363a9ada93 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:32.392635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1006/2002] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:32 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:32.396797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f91220-f149-479c-8f1f-b764b98f85e2 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:32.409153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f91220-f149-479c-8f1f-b764b98f85e2 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:32.416883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acc382-8e39-47ac-9bd0-ceaedd0cf7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:32.417586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acc382-8e39-47ac-9bd0-ceaedd0cf7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:32.417880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acc382-8e39-47ac-9bd0-ceaedd0cf7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:32.418164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acc382-8e39-47ac-9bd0-ceaedd0cf7c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:32.418912 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acc382-8e39-47ac-9bd0-ceaedd0cf7c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:32.420635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acc382-8e39-47ac-9bd0-ceaedd0cf7c4 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:32.436496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 997/2003] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:32.437926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6703b-6e0e-4000-9fbe-d5363a9ada93 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:32.448777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6703b-6e0e-4000-9fbe-d5363a9ada93 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:32.453861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d1231-7ebe-4d30-a604-e1eaae635a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:32.460744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d1231-7ebe-4d30-a604-e1eaae635a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:32.461739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d1231-7ebe-4d30-a604-e1eaae635a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:32.462128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d1231-7ebe-4d30-a604-e1eaae635a12 admin admin] PATH_INFO: `/v3/projects/e22dfc634790498e88936c0b8a7c1fc2/users/32ecd339ba12462d88778d592f07258e/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:32.488983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d1231-7ebe-4d30-a604-e1eaae635a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e22dfc634790498e88936c0b8a7c1fc2, user_id=32ecd339ba12462d88778d592f07258e, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:32.491750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d1231-7ebe-4d30-a604-e1eaae635a12 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:32.503864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1007/2004] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:32 2022] PUT /identity/v3/projects/e22dfc634790498e88936c0b8a7c1fc2/users/32ecd339ba12462d88778d592f07258e/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:32.507790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acc382-8e39-47ac-9bd0-ceaedd0cf7c4 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:32.517842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acc382-8e39-47ac-9bd0-ceaedd0cf7c4 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:32.562461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:32.576339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0bafc-bb74-4393-9744-ac80e9ce9220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:32.577006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0bafc-bb74-4393-9744-ac80e9ce9220 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:32.577290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0bafc-bb74-4393-9744-ac80e9ce9220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:32.577566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0bafc-bb74-4393-9744-ac80e9ce9220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:32.578278 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0bafc-bb74-4393-9744-ac80e9ce9220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:32.579862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0bafc-bb74-4393-9744-ac80e9ce9220 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:32.590096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 998/2005] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:32.594142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d1231-7ebe-4d30-a604-e1eaae635a12 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:32.605008 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d1231-7ebe-4d30-a604-e1eaae635a12 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:32.609525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80812ae-58d6-4700-93b9-4bbf56b9a148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:32.610395 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80812ae-58d6-4700-93b9-4bbf56b9a148 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:32.610686 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80812ae-58d6-4700-93b9-4bbf56b9a148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:32.610978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80812ae-58d6-4700-93b9-4bbf56b9a148 admin admin] PATH_INFO: `/v3/projects/e22dfc634790498e88936c0b8a7c1fc2/users/32ecd339ba12462d88778d592f07258e/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:32.612956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80812ae-58d6-4700-93b9-4bbf56b9a148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e22dfc634790498e88936c0b8a7c1fc2, user_id=32ecd339ba12462d88778d592f07258e, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:32.615573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80812ae-58d6-4700-93b9-4bbf56b9a148 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:32.630531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1008/2006] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:32 2022] PUT /identity/v3/projects/e22dfc634790498e88936c0b8a7c1fc2/users/32ecd339ba12462d88778d592f07258e/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:32.641533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149e8de-8d69-4822-9502-a1130fad7c32 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:32.641895 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149e8de-8d69-4822-9502-a1130fad7c32 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:32.642331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149e8de-8d69-4822-9502-a1130fad7c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:32.678279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-b149e8de-8d69-4822-9502-a1130fad7c32 None None] MFA Rules not processed for user `32ecd339ba12462d88778d592f07258e`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:32.680297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b149e8de-8d69-4822-9502-a1130fad7c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:32.788829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 999/2007] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:32.799872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80812ae-58d6-4700-93b9-4bbf56b9a148 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:32.871493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80812ae-58d6-4700-93b9-4bbf56b9a148 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:32.876018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7a6ec-b8db-466d-bbff-a7eceb43390b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:32.876825 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7a6ec-b8db-466d-bbff-a7eceb43390b service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:32.877012 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7a6ec-b8db-466d-bbff-a7eceb43390b service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:32.877633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7a6ec-b8db-466d-bbff-a7eceb43390b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:32.937742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:32.948961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7a6ec-b8db-466d-bbff-a7eceb43390b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:32.950509 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7a6ec-b8db-466d-bbff-a7eceb43390b service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:32.960029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1009/2008] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:33.685451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149e8de-8d69-4822-9502-a1130fad7c32 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:33.694716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149e8de-8d69-4822-9502-a1130fad7c32 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:33.698221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4ee8c-8f2d-4f87-bfd6-f8546b11d051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:33.698854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4ee8c-8f2d-4f87-bfd6-f8546b11d051 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:33.699137 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4ee8c-8f2d-4f87-bfd6-f8546b11d051 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:33.699413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4ee8c-8f2d-4f87-bfd6-f8546b11d051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:33.701153 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b0c4ee8c-8f2d-4f87-bfd6-f8546b11d051 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:33.764930 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4ee8c-8f2d-4f87-bfd6-f8546b11d051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:33.766377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4ee8c-8f2d-4f87-bfd6-f8546b11d051 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:33.775341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1000/2009] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:34.408851 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7a6ec-b8db-466d-bbff-a7eceb43390b service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:34.418919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7a6ec-b8db-466d-bbff-a7eceb43390b service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:34.422940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b84518-7091-46d7-a005-81d53e8cae71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:34.424037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b84518-7091-46d7-a005-81d53e8cae71 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:34.424037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b84518-7091-46d7-a005-81d53e8cae71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:34.424257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b84518-7091-46d7-a005-81d53e8cae71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:34.499794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b84518-7091-46d7-a005-81d53e8cae71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:34.501316 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b84518-7091-46d7-a005-81d53e8cae71 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:34.552086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1010/2010] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:35.814833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4ee8c-8f2d-4f87-bfd6-f8546b11d051 service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:35.985540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4ee8c-8f2d-4f87-bfd6-f8546b11d051 service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:35.987472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b84518-7091-46d7-a005-81d53e8cae71 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:35.999752 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b84518-7091-46d7-a005-81d53e8cae71 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:36.002237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:36.003530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:36.004756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:36.005260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] PATH_INFO: `/v3/users/357ece69b4ec4b1383cd75b205c25879` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:36.006821 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=357ece69b4ec4b1383cd75b205c25879)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:36.009549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5073fbd-dd9a-48fc-bcf2-2f42e876be79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:36.009549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5073fbd-dd9a-48fc-bcf2-2f42e876be79 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:36.009549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5073fbd-dd9a-48fc-bcf2-2f42e876be79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:36.010020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5073fbd-dd9a-48fc-bcf2-2f42e876be79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:36.012164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:36.019805 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:36.021201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c5073fbd-dd9a-48fc-bcf2-2f42e876be79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:36.106691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '357ece69b4ec4b1383cd75b205c25879'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:36.111413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '357ece69b4ec4b1383cd75b205c25879'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:36.123142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '357ece69b4ec4b1383cd75b205c25879'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:36.145184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '357ece69b4ec4b1383cd75b205c25879'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:36.146205 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '357ece69b4ec4b1383cd75b205c25879'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:36.155081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5073fbd-dd9a-48fc-bcf2-2f42e876be79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:36.157161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5073fbd-dd9a-48fc-bcf2-2f42e876be79 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:36.168233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1011/2011] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:20:36.176852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1001/2012] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:35 2022] DELETE /identity/v3/users/357ece69b4ec4b1383cd75b205c25879 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:36.182091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cad41-216a-4fc1-abfa-0c5225cd77e4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:36.182453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cad41-216a-4fc1-abfa-0c5225cd77e4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:36.182790 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cad41-216a-4fc1-abfa-0c5225cd77e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:36.205803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-3b0cad41-216a-4fc1-abfa-0c5225cd77e4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:36.207073 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3b0cad41-216a-4fc1-abfa-0c5225cd77e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:36.355687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1012/2013] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:36.368394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:36.369957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:36.522571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f6351-a6a8-4be2-81e7-ac89008b84f3 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:36.527544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355240c-3df1-4c56-bea4-a56ee5151526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:36.528266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355240c-3df1-4c56-bea4-a56ee5151526 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:36.528610 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355240c-3df1-4c56-bea4-a56ee5151526 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:36.528895 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355240c-3df1-4c56-bea4-a56ee5151526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:36.543296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355240c-3df1-4c56-bea4-a56ee5151526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:36.545025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355240c-3df1-4c56-bea4-a56ee5151526 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:36.560430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1002/2014] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:20:37.306251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cad41-216a-4fc1-abfa-0c5225cd77e4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:37.307724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3b0cad41-216a-4fc1-abfa-0c5225cd77e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:37.321752 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cad41-216a-4fc1-abfa-0c5225cd77e4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:37.331905 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:37.332744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:37.333033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:37.333314 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] PATH_INFO: `/v3/projects/f82081e918504de7b3505338d7d8c22f` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:37.336766 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f82081e918504de7b3505338d7d8c22f)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:37.361730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:37.401848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:37.435400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f82081e918504de7b3505338d7d8c22f', 'request_id': 'req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:37.443383 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f82081e918504de7b3505338d7d8c22f', 'request_id': 'req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:37.451473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f82081e918504de7b3505338d7d8c22f', 'request_id': 'req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:37.470517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] The token cache is being invalidate because project f82081e918504de7b3505338d7d8c22f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:37.470818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:37.471678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:37.473743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1013/2015] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:37 2022] DELETE /identity/v3/projects/f82081e918504de7b3505338d7d8c22f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:37.477621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355240c-3df1-4c56-bea4-a56ee5151526 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:37.491650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355240c-3df1-4c56-bea4-a56ee5151526 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:37.604025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8313ff1-a19a-49db-8e28-80e6fb65a19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:37.605337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8313ff1-a19a-49db-8e28-80e6fb65a19e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:37.605626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8313ff1-a19a-49db-8e28-80e6fb65a19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:37.605911 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8313ff1-a19a-49db-8e28-80e6fb65a19e admin admin] PATH_INFO: `/v3/users/32ecd339ba12462d88778d592f07258e` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:37.608718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8313ff1-a19a-49db-8e28-80e6fb65a19e admin admin] RBAC: Authorizing `identity:delete_user(user_id=32ecd339ba12462d88778d592f07258e)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:37.610910 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8313ff1-a19a-49db-8e28-80e6fb65a19e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:37.671621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:37.694293 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b8313ff1-a19a-49db-8e28-80e6fb65a19e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32ecd339ba12462d88778d592f07258e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:37.695644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b8313ff1-a19a-49db-8e28-80e6fb65a19e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32ecd339ba12462d88778d592f07258e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:37.705845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b8313ff1-a19a-49db-8e28-80e6fb65a19e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32ecd339ba12462d88778d592f07258e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:37.776754 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b8313ff1-a19a-49db-8e28-80e6fb65a19e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32ecd339ba12462d88778d592f07258e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:37.779284 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-b8313ff1-a19a-49db-8e28-80e6fb65a19e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32ecd339ba12462d88778d592f07258e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:37.813096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1003/2016] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:37 2022] DELETE /identity/v3/users/32ecd339ba12462d88778d592f07258e => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:38.503202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:38.504722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:38.547231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0e67d-3efd-4bfc-b28a-a74d8232c6a2 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:38.729292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:38.730194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:38.730658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:38.731093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] PATH_INFO: `/v3/projects/e22dfc634790498e88936c0b8a7c1fc2` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:38.734741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e22dfc634790498e88936c0b8a7c1fc2)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:38.738710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:38.876408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e22dfc634790498e88936c0b8a7c1fc2', 'request_id': 'req-5adb3ba0-3c12-44a1-8080-0746447202c7'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:38.909058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e22dfc634790498e88936c0b8a7c1fc2', 'request_id': 'req-5adb3ba0-3c12-44a1-8080-0746447202c7'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:38.936620 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e22dfc634790498e88936c0b8a7c1fc2', 'request_id': 'req-5adb3ba0-3c12-44a1-8080-0746447202c7'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:38.975627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] The token cache is being invalidate because project e22dfc634790498e88936c0b8a7c1fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:38.975928 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:38.978526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:38.981222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1014/2017] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:38 2022] DELETE /identity/v3/projects/e22dfc634790498e88936c0b8a7c1fc2 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:38.987000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6b310-b6cd-4881-a317-e5058a9bfa2d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:38.987533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6b310-b6cd-4881-a317-e5058a9bfa2d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:38.987533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6b310-b6cd-4881-a317-e5058a9bfa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.029266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-d6a6b310-b6cd-4881-a317-e5058a9bfa2d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:39.030512 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d6a6b310-b6cd-4881-a317-e5058a9bfa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:39.067091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:39.068551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:39.180508 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1004/2018] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:39.189443 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:39.191623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d2003-92a9-4ee9-b321-826a611b6277 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.191928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d2003-92a9-4ee9-b321-826a611b6277 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.192250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d2003-92a9-4ee9-b321-826a611b6277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.225704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb3ba0-3c12-44a1-8080-0746447202c7 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:39.226967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-a66d2003-92a9-4ee9-b321-826a611b6277 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:39.228215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a66d2003-92a9-4ee9-b321-826a611b6277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:39.233388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:39.234738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.235199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.235629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] PATH_INFO: `/v3/users/9262f80f454a4721beaff65e97f6b55d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.237275 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9262f80f454a4721beaff65e97f6b55d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:39.239709 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:39.245781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1005/2019] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:39.246616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d2003-92a9-4ee9-b321-826a611b6277 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:39.248881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a66d2003-92a9-4ee9-b321-826a611b6277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:39.263527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d2003-92a9-4ee9-b321-826a611b6277 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:39.269677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cdb7a-73a5-4857-8520-adbf7a6f2845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:39.270257 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cdb7a-73a5-4857-8520-adbf7a6f2845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.270543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cdb7a-73a5-4857-8520-adbf7a6f2845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.270821 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cdb7a-73a5-4857-8520-adbf7a6f2845 admin admin] PATH_INFO: `/v3/users/9902a690b63d48989c8325ea0b80ae3c` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.272232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cdb7a-73a5-4857-8520-adbf7a6f2845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9902a690b63d48989c8325ea0b80ae3c)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:39.274213 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cdb7a-73a5-4857-8520-adbf7a6f2845 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:39.324816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9262f80f454a4721beaff65e97f6b55d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:39.337145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9262f80f454a4721beaff65e97f6b55d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:39.338136 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9262f80f454a4721beaff65e97f6b55d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:39.353819 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f41cdb7a-73a5-4857-8520-adbf7a6f2845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9902a690b63d48989c8325ea0b80ae3c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:39.354334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9262f80f454a4721beaff65e97f6b55d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:39.354525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f41cdb7a-73a5-4857-8520-adbf7a6f2845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9902a690b63d48989c8325ea0b80ae3c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:39.355224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9262f80f454a4721beaff65e97f6b55d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:39.373376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f41cdb7a-73a5-4857-8520-adbf7a6f2845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9902a690b63d48989c8325ea0b80ae3c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:39.374270 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1015/2020] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:39 2022] DELETE /identity/v3/users/9262f80f454a4721beaff65e97f6b55d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:39.375286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:39.377606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:39.405416 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f41cdb7a-73a5-4857-8520-adbf7a6f2845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9902a690b63d48989c8325ea0b80ae3c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:39.406280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f41cdb7a-73a5-4857-8520-adbf7a6f2845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9902a690b63d48989c8325ea0b80ae3c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:39.430776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1006/2021] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:39 2022] DELETE /identity/v3/users/9902a690b63d48989c8325ea0b80ae3c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:39.432457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3138906-e5df-44be-b8eb-24851a3dd570 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.433348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3138906-e5df-44be-b8eb-24851a3dd570 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.433700 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3138906-e5df-44be-b8eb-24851a3dd570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.462454 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f3138906-e5df-44be-b8eb-24851a3dd570 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:39.463703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f3138906-e5df-44be-b8eb-24851a3dd570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:39.472678 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:39.477524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0fac4-3563-4108-8c3d-221e46362c9d admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:39.544958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1007/2022] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:39.546402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e6c59-b895-46cf-b425-d17c0c753a86 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.546695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e6c59-b895-46cf-b425-d17c0c753a86 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.546976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e6c59-b895-46cf-b425-d17c0c753a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.568977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e679d-fe2c-4e2e-a67e-225bfaba2904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:39.569993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e679d-fe2c-4e2e-a67e-225bfaba2904 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.570400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e679d-fe2c-4e2e-a67e-225bfaba2904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.570838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e679d-fe2c-4e2e-a67e-225bfaba2904 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.572618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e679d-fe2c-4e2e-a67e-225bfaba2904 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:39.573044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e679d-fe2c-4e2e-a67e-225bfaba2904 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:39.575428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e679d-fe2c-4e2e-a67e-225bfaba2904 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:39.576473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-fc3e6c59-b895-46cf-b425-d17c0c753a86 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:39.577689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-fc3e6c59-b895-46cf-b425-d17c0c753a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:39.583849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:39.588597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1008/2023] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:39.588597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e6c59-b895-46cf-b425-d17c0c753a86 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:39.590371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-fc3e6c59-b895-46cf-b425-d17c0c753a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:39.605199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1016/2024] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:20:39.607037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e679d-fe2c-4e2e-a67e-225bfaba2904 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:39.625049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e679d-fe2c-4e2e-a67e-225bfaba2904 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:39.629979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecaccd9-f4fa-4b14-bb97-797562da8899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:39.631442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecaccd9-f4fa-4b14-bb97-797562da8899 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.631740 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecaccd9-f4fa-4b14-bb97-797562da8899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.632019 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecaccd9-f4fa-4b14-bb97-797562da8899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.632750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecaccd9-f4fa-4b14-bb97-797562da8899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:39.634715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecaccd9-f4fa-4b14-bb97-797562da8899 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:39.658417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e6c59-b895-46cf-b425-d17c0c753a86 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:39.662677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ebf0f-4087-42bc-bfde-282a38af4cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:39.663463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ebf0f-4087-42bc-bfde-282a38af4cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.663752 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ebf0f-4087-42bc-bfde-282a38af4cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.664572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ebf0f-4087-42bc-bfde-282a38af4cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.674849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1017/2025] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:39 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:39.674849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecaccd9-f4fa-4b14-bb97-797562da8899 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:39.675237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ebf0f-4087-42bc-bfde-282a38af4cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:39.675890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5ecaccd9-f4fa-4b14-bb97-797562da8899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:39.676775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ebf0f-4087-42bc-bfde-282a38af4cc4 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:39.688400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1009/2026] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:39.689450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ebf0f-4087-42bc-bfde-282a38af4cc4 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:39.701485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ebf0f-4087-42bc-bfde-282a38af4cc4 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:39.763132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecaccd9-f4fa-4b14-bb97-797562da8899 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:39.769964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc2ec8-bd37-4323-91db-9bdf06d65d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:39.770973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc2ec8-bd37-4323-91db-9bdf06d65d93 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.771334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc2ec8-bd37-4323-91db-9bdf06d65d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.772084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc2ec8-bd37-4323-91db-9bdf06d65d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.780090 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913b1f8-402d-4f74-b42c-5e258cffccc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:39.780781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913b1f8-402d-4f74-b42c-5e258cffccc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.781206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913b1f8-402d-4f74-b42c-5e258cffccc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.781491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913b1f8-402d-4f74-b42c-5e258cffccc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.782191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913b1f8-402d-4f74-b42c-5e258cffccc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:39.785159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913b1f8-402d-4f74-b42c-5e258cffccc0 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:39.817872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1010/2027] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:39 2022] POST /identity/v3/users => generated 285 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:39.819589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913b1f8-402d-4f74-b42c-5e258cffccc0 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:39.841211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913b1f8-402d-4f74-b42c-5e258cffccc0 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:39.847917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d21608-e518-4d26-9e9a-3d2944d269a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:39.849190 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d21608-e518-4d26-9e9a-3d2944d269a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.849872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d21608-e518-4d26-9e9a-3d2944d269a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.850262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d21608-e518-4d26-9e9a-3d2944d269a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.851390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d21608-e518-4d26-9e9a-3d2944d269a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:39.854642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d21608-e518-4d26-9e9a-3d2944d269a2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:39.880422 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1011/2028] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:39.884225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d21608-e518-4d26-9e9a-3d2944d269a2 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:39.895444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d21608-e518-4d26-9e9a-3d2944d269a2 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:39.914749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53267f8-816f-40a3-816d-e350a7e4f757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:39.916305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53267f8-816f-40a3-816d-e350a7e4f757 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.917258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc2ec8-bd37-4323-91db-9bdf06d65d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:39.919465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53267f8-816f-40a3-816d-e350a7e4f757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.919465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53267f8-816f-40a3-816d-e350a7e4f757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.920023 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53267f8-816f-40a3-816d-e350a7e4f757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:39.922227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53267f8-816f-40a3-816d-e350a7e4f757 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:39.923629 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc2ec8-bd37-4323-91db-9bdf06d65d93 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:39.938763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1018/2029] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:39.944651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1012/2030] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:39.946281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc2ec8-bd37-4323-91db-9bdf06d65d93 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:39.962610 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc2ec8-bd37-4323-91db-9bdf06d65d93 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:39.968189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce895b-02b5-4f06-a5c3-8d2adde384dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:39.969295 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce895b-02b5-4f06-a5c3-8d2adde384dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:39.969645 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce895b-02b5-4f06-a5c3-8d2adde384dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:39.970551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce895b-02b5-4f06-a5c3-8d2adde384dc admin admin] PATH_INFO: `/v3/projects/1a6acf47bea7483b81aeda9070e69537/users/0d095d34fd104238b9b75c9950d276af/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:39.989658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce895b-02b5-4f06-a5c3-8d2adde384dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a6acf47bea7483b81aeda9070e69537, user_id=0d095d34fd104238b9b75c9950d276af, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:39.992879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce895b-02b5-4f06-a5c3-8d2adde384dc admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:40.017706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1019/2031] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:39 2022] PUT /identity/v3/projects/1a6acf47bea7483b81aeda9070e69537/users/0d095d34fd104238b9b75c9950d276af/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:40.028265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed2f15-5b23-4c44-922f-af23d895fd84 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:40.028265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed2f15-5b23-4c44-922f-af23d895fd84 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:40.028899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed2f15-5b23-4c44-922f-af23d895fd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:40.069285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-11ed2f15-5b23-4c44-922f-af23d895fd84 None None] MFA Rules not processed for user `0d095d34fd104238b9b75c9950d276af`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:40.070704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-11ed2f15-5b23-4c44-922f-af23d895fd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:40.085154 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:40.226305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1013/2032] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:40 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:40.234765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce895b-02b5-4f06-a5c3-8d2adde384dc admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:40.237894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f7ce895b-02b5-4f06-a5c3-8d2adde384dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:40.307686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed2f15-5b23-4c44-922f-af23d895fd84 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:40.318299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-11ed2f15-5b23-4c44-922f-af23d895fd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:40.380917 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce895b-02b5-4f06-a5c3-8d2adde384dc admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:40.390806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955dfbde-f3b2-4332-be4b-7201d84e42fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48586ff1269f49a3ae3d9f3e40c1b9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:40.393272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dfbde-f3b2-4332-be4b-7201d84e42fe service glance] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:40.393771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dfbde-f3b2-4332-be4b-7201d84e42fe service glance] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:40.394210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dfbde-f3b2-4332-be4b-7201d84e42fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:40.403047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed2f15-5b23-4c44-922f-af23d895fd84 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:40.407420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955dfbde-f3b2-4332-be4b-7201d84e42fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:40.408402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:40.408911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955dfbde-f3b2-4332-be4b-7201d84e42fe service glance] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:40.409726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:40.410441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:40.410732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] PATH_INFO: `/v3/projects/cff1ab65510843838f21195007737319` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:40.412672 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff1ab65510843838f21195007737319)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:40.415027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:40.421153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1020/2033] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:40 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:40.503307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff1ab65510843838f21195007737319', 'request_id': 'req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.512884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955dfbde-f3b2-4332-be4b-7201d84e42fe service glance] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:40.513953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff1ab65510843838f21195007737319', 'request_id': 'req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.520556 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-955dfbde-f3b2-4332-be4b-7201d84e42fe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:40.531424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff1ab65510843838f21195007737319', 'request_id': 'req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.546801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] The token cache is being invalidate because project cff1ab65510843838f21195007737319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:40.547198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.547926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.560480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1014/2034] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:40 2022] DELETE /identity/v3/projects/cff1ab65510843838f21195007737319 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:40.565139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:40.568960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:40.573174 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:40.586142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955dfbde-f3b2-4332-be4b-7201d84e42fe service glance] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:40.668662 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:40.721789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:40.722894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:40.723421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:40.723695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] PATH_INFO: `/v3/projects/16e369c52cb54dbfb5704e70e2c47b00` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:40.725163 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] RBAC: Authorizing `identity:delete_project(project_id=16e369c52cb54dbfb5704e70e2c47b00)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:40.725510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c389d3-cbe1-48bc-8a6f-ea3b04191a4e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:40.727501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:40.731429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:40.732428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:40.732791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:40.733145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] PATH_INFO: `/v3/users/246fee748ecd414cbef678442620dd5f` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:40.734490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=246fee748ecd414cbef678442620dd5f)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:40.741585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:40.806160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '246fee748ecd414cbef678442620dd5f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.806842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '246fee748ecd414cbef678442620dd5f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.807408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16e369c52cb54dbfb5704e70e2c47b00', 'request_id': 'req-541abe24-a468-41e0-ac94-ffee32f7f07f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.815835 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '246fee748ecd414cbef678442620dd5f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.821167 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16e369c52cb54dbfb5704e70e2c47b00', 'request_id': 'req-541abe24-a468-41e0-ac94-ffee32f7f07f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.829396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16e369c52cb54dbfb5704e70e2c47b00', 'request_id': 'req-541abe24-a468-41e0-ac94-ffee32f7f07f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.832595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '246fee748ecd414cbef678442620dd5f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.833516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '246fee748ecd414cbef678442620dd5f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.843474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] The token cache is being invalidate because project 16e369c52cb54dbfb5704e70e2c47b00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:40.843774 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.844511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:40.846166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1021/2035] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:40 2022] DELETE /identity/v3/projects/16e369c52cb54dbfb5704e70e2c47b00 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:40.849116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1015/2036] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:40 2022] DELETE /identity/v3/users/246fee748ecd414cbef678442620dd5f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:40.850098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:40.851684 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:40.941108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541abe24-a468-41e0-ac94-ffee32f7f07f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:41.001979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:41.025967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:41.027241 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:41.027525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:41.027802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] PATH_INFO: `/v3/users/61cd93a17e8246ad88caec87616ad594` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:41.029521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=61cd93a17e8246ad88caec87616ad594)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:41.031961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:41.128750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61cd93a17e8246ad88caec87616ad594'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.142774 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61cd93a17e8246ad88caec87616ad594'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.143434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61cd93a17e8246ad88caec87616ad594'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.157228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61cd93a17e8246ad88caec87616ad594'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.157980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61cd93a17e8246ad88caec87616ad594'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.172479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1022/2037] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:40 2022] DELETE /identity/v3/users/61cd93a17e8246ad88caec87616ad594 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:41.277438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:41.279035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:41.313803 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a2a3b-722b-45a5-bb96-b9ba3d351c30 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:41.442970 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:41.444241 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:41.444526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:41.444880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] PATH_INFO: `/v3/projects/3a8c2e4c7ebf4d57998f55034e87fc30` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:41.446446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a8c2e4c7ebf4d57998f55034e87fc30)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:41.448515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:41.552225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a8c2e4c7ebf4d57998f55034e87fc30', 'request_id': 'req-c877860c-ceb8-4524-92b2-85db528e61fb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.575578 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a8c2e4c7ebf4d57998f55034e87fc30', 'request_id': 'req-c877860c-ceb8-4524-92b2-85db528e61fb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.579772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:41.592772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a8c2e4c7ebf4d57998f55034e87fc30', 'request_id': 'req-c877860c-ceb8-4524-92b2-85db528e61fb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.612152 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] The token cache is being invalidate because project 3a8c2e4c7ebf4d57998f55034e87fc30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:41.612446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.613134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.615062 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1016/2038] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:41 2022] DELETE /identity/v3/projects/3a8c2e4c7ebf4d57998f55034e87fc30 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:41.621369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:41.623049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:41.636369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ea963-cc8f-4126-8090-81a10c7c7d8f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:41.723117 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:41.724106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:41.724476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:41.725088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] PATH_INFO: `/v3/users/d14fbc32b79e41d1ba01b29141d93865` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:41.726509 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d14fbc32b79e41d1ba01b29141d93865)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:41.730081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:41.781840 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:41.784128 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:41.801053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877860c-ceb8-4524-92b2-85db528e61fb admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:41.806588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:41.807548 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:41.807924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:41.808402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] PATH_INFO: `/v3/projects/d5f0d1d0426a4ec1b88f835c132b1f97` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:41.809908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5f0d1d0426a4ec1b88f835c132b1f97)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:41.811563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd14fbc32b79e41d1ba01b29141d93865'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.812402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:41.825775 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd14fbc32b79e41d1ba01b29141d93865'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.828520 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd14fbc32b79e41d1ba01b29141d93865'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.845305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:41.854000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd14fbc32b79e41d1ba01b29141d93865'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.856176 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd14fbc32b79e41d1ba01b29141d93865'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.874711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1023/2039] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:41 2022] DELETE /identity/v3/users/d14fbc32b79e41d1ba01b29141d93865 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:41.882613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5f0d1d0426a4ec1b88f835c132b1f97', 'request_id': 'req-63025a22-2d53-41b7-a126-7b6ce0345eb2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.891040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5f0d1d0426a4ec1b88f835c132b1f97', 'request_id': 'req-63025a22-2d53-41b7-a126-7b6ce0345eb2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.898254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5f0d1d0426a4ec1b88f835c132b1f97', 'request_id': 'req-63025a22-2d53-41b7-a126-7b6ce0345eb2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.909253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] The token cache is being invalidate because project d5f0d1d0426a4ec1b88f835c132b1f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:41.909542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.910219 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:41.912035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1017/2040] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:41 2022] DELETE /identity/v3/projects/d5f0d1d0426a4ec1b88f835c132b1f97 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:41.916037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:41.917558 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:41.990495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f6a1f-6db8-42ed-bded-2f5fea0c6f31 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:41.996960 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:41.997877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:41.998211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:41.998725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] PATH_INFO: `/v3/users/fe22ded872ab4ed893216296cfea091f` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:42.000360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe22ded872ab4ed893216296cfea091f)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:42.002691 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:42.060568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe22ded872ab4ed893216296cfea091f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.068946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe22ded872ab4ed893216296cfea091f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.069771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe22ded872ab4ed893216296cfea091f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.086755 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe22ded872ab4ed893216296cfea091f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.087678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe22ded872ab4ed893216296cfea091f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.102530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1024/2041] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:41 2022] DELETE /identity/v3/users/fe22ded872ab4ed893216296cfea091f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:42.377924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:42.379771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:42.461092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:42.463597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:42.477452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63025a22-2d53-41b7-a126-7b6ce0345eb2 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:42.542013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4d528-e318-4a71-9e98-e1c6c96ebb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:42.542888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4d528-e318-4a71-9e98-e1c6c96ebb94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:42.543168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4d528-e318-4a71-9e98-e1c6c96ebb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:42.543535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4d528-e318-4a71-9e98-e1c6c96ebb94 admin admin] PATH_INFO: `/v3/projects/e3811a42b04e48889fc7577cdff28816` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:42.543721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07767ea-0357-4019-980b-8192eac93c7a admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:42.544781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4d528-e318-4a71-9e98-e1c6c96ebb94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3811a42b04e48889fc7577cdff28816)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:42.546684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4d528-e318-4a71-9e98-e1c6c96ebb94 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:42.548376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:42.549334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:42.549680 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:42.550021 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc admin admin] PATH_INFO: `/v3/projects/d6820e7741244585b8dbc9980d7d605f` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:42.551294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6820e7741244585b8dbc9980d7d605f)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:42.553290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:42.556359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:42.573101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:42.601205 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-40c4d528-e318-4a71-9e98-e1c6c96ebb94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3811a42b04e48889fc7577cdff28816', 'request_id': 'req-40c4d528-e318-4a71-9e98-e1c6c96ebb94'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.604435 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6820e7741244585b8dbc9980d7d605f', 'request_id': 'req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.609622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-40c4d528-e318-4a71-9e98-e1c6c96ebb94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3811a42b04e48889fc7577cdff28816', 'request_id': 'req-40c4d528-e318-4a71-9e98-e1c6c96ebb94'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.610438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6820e7741244585b8dbc9980d7d605f', 'request_id': 'req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.617768 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6820e7741244585b8dbc9980d7d605f', 'request_id': 'req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.622337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-40c4d528-e318-4a71-9e98-e1c6c96ebb94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3811a42b04e48889fc7577cdff28816', 'request_id': 'req-40c4d528-e318-4a71-9e98-e1c6c96ebb94'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.631926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc admin admin] The token cache is being invalidate because project d6820e7741244585b8dbc9980d7d605f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:42.632533 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.633024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4f425985-ad7a-4e7f-af3a-66eb8acf77dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.633540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-40c4d528-e318-4a71-9e98-e1c6c96ebb94 admin admin] The token cache is being invalidate because project e3811a42b04e48889fc7577cdff28816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:42.633826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-40c4d528-e318-4a71-9e98-e1c6c96ebb94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.634461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-40c4d528-e318-4a71-9e98-e1c6c96ebb94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.634567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1025/2042] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:42 2022] DELETE /identity/v3/projects/d6820e7741244585b8dbc9980d7d605f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:42.636003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1018/2043] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:42 2022] DELETE /identity/v3/projects/e3811a42b04e48889fc7577cdff28816 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:42.640540 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5546b62-7f44-402d-b855-34f24e9e9c6a None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:42.640831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5546b62-7f44-402d-b855-34f24e9e9c6a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:42.641129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5546b62-7f44-402d-b855-34f24e9e9c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:42.661467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-a5546b62-7f44-402d-b855-34f24e9e9c6a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:42.662980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-a5546b62-7f44-402d-b855-34f24e9e9c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:42.737533 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1026/2044] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:42.748312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e7d26-391d-4346-867a-31fce74423e2 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:42.748523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e7d26-391d-4346-867a-31fce74423e2 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:42.748860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e7d26-391d-4346-867a-31fce74423e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:42.767918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3a3e7d26-391d-4346-867a-31fce74423e2 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:42.769259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3a3e7d26-391d-4346-867a-31fce74423e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:42.782359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1019/2045] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:42.795196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5546b62-7f44-402d-b855-34f24e9e9c6a None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:42.806512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5546b62-7f44-402d-b855-34f24e9e9c6a None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:42.813522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b563f9-a9d7-4e81-97fb-1d1485a30367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:42.815048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b563f9-a9d7-4e81-97fb-1d1485a30367 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:42.815048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b563f9-a9d7-4e81-97fb-1d1485a30367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:42.815048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b563f9-a9d7-4e81-97fb-1d1485a30367 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:42.815804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b563f9-a9d7-4e81-97fb-1d1485a30367 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:42.816160 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b563f9-a9d7-4e81-97fb-1d1485a30367 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:42.817813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b563f9-a9d7-4e81-97fb-1d1485a30367 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:42.831293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1027/2046] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:20:42.835579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e7d26-391d-4346-867a-31fce74423e2 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:42.845736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e7d26-391d-4346-867a-31fce74423e2 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:42.850428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fc4b8-39a7-48ae-a61f-5be41766ae17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:42.859785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fc4b8-39a7-48ae-a61f-5be41766ae17 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:42.860126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fc4b8-39a7-48ae-a61f-5be41766ae17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:42.860415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fc4b8-39a7-48ae-a61f-5be41766ae17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:42.861140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fc4b8-39a7-48ae-a61f-5be41766ae17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:42.863057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fc4b8-39a7-48ae-a61f-5be41766ae17 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:42.881338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b563f9-a9d7-4e81-97fb-1d1485a30367 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:42.883792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-42b563f9-a9d7-4e81-97fb-1d1485a30367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:42.900328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b563f9-a9d7-4e81-97fb-1d1485a30367 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:42.905672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd9f30-3981-4ebb-9158-9ef81604a6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:42.906833 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd9f30-3981-4ebb-9158-9ef81604a6a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:42.907029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd9f30-3981-4ebb-9158-9ef81604a6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:42.907135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1020/2047] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:42 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:42.907350 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd9f30-3981-4ebb-9158-9ef81604a6a8 admin admin] PATH_INFO: `/v3/users/d8e2b6536c48498db1c3cf97b141ab6f` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:42.907981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fc4b8-39a7-48ae-a61f-5be41766ae17 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:42.908716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd9f30-3981-4ebb-9158-9ef81604a6a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e2b6536c48498db1c3cf97b141ab6f)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:42.910745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd9f30-3981-4ebb-9158-9ef81604a6a8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:42.971817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-20dd9f30-3981-4ebb-9158-9ef81604a6a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e2b6536c48498db1c3cf97b141ab6f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.980740 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-20dd9f30-3981-4ebb-9158-9ef81604a6a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e2b6536c48498db1c3cf97b141ab6f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.981461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-20dd9f30-3981-4ebb-9158-9ef81604a6a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e2b6536c48498db1c3cf97b141ab6f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.983404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fc4b8-39a7-48ae-a61f-5be41766ae17 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:42.988480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27aacc-590d-4eac-82eb-2b3d468f3f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:42.989158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27aacc-590d-4eac-82eb-2b3d468f3f78 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:42.989472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27aacc-590d-4eac-82eb-2b3d468f3f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:42.989764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27aacc-590d-4eac-82eb-2b3d468f3f78 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:42.990460 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27aacc-590d-4eac-82eb-2b3d468f3f78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:42.992437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27aacc-590d-4eac-82eb-2b3d468f3f78 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:42.994410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-20dd9f30-3981-4ebb-9158-9ef81604a6a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e2b6536c48498db1c3cf97b141ab6f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.995336 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-20dd9f30-3981-4ebb-9158-9ef81604a6a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e2b6536c48498db1c3cf97b141ab6f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:42.996570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:43.011168 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1028/2048] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:42 2022] DELETE /identity/v3/users/d8e2b6536c48498db1c3cf97b141ab6f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:43.015110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1021/2049] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:42 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:43.015581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2bd4c-d9d2-4ae6-b2e1-c5b6a882496c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:43.015948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2bd4c-d9d2-4ae6-b2e1-c5b6a882496c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:43.016433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2bd4c-d9d2-4ae6-b2e1-c5b6a882496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:43.019104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27aacc-590d-4eac-82eb-2b3d468f3f78 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:43.020821 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-db27aacc-590d-4eac-82eb-2b3d468f3f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:43.033260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27aacc-590d-4eac-82eb-2b3d468f3f78 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:43.035268 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-fff2bd4c-d9d2-4ae6-b2e1-c5b6a882496c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:43.036501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-fff2bd4c-d9d2-4ae6-b2e1-c5b6a882496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:43.093985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4ad30-3d22-4065-bbb5-6f39db9edb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:43.094697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4ad30-3d22-4065-bbb5-6f39db9edb85 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:43.094984 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4ad30-3d22-4065-bbb5-6f39db9edb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:43.095352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4ad30-3d22-4065-bbb5-6f39db9edb85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:43.096096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4ad30-3d22-4065-bbb5-6f39db9edb85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:43.097809 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4ad30-3d22-4065-bbb5-6f39db9edb85 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:43.098030 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1029/2050] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:43.108602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2bd4c-d9d2-4ae6-b2e1-c5b6a882496c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:43.110224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-fff2bd4c-d9d2-4ae6-b2e1-c5b6a882496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:43.112687 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1022/2051] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:43.116381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4ad30-3d22-4065-bbb5-6f39db9edb85 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:43.123651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:43.128367 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4ad30-3d22-4065-bbb5-6f39db9edb85 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:43.137985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337fcf6-4106-4dba-8e5f-1c339cecdca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:43.138976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337fcf6-4106-4dba-8e5f-1c339cecdca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:43.139454 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337fcf6-4106-4dba-8e5f-1c339cecdca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:43.139936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337fcf6-4106-4dba-8e5f-1c339cecdca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:43.140770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337fcf6-4106-4dba-8e5f-1c339cecdca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:43.142390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337fcf6-4106-4dba-8e5f-1c339cecdca5 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:43.153696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1023/2052] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:43.157421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337fcf6-4106-4dba-8e5f-1c339cecdca5 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:43.170347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337fcf6-4106-4dba-8e5f-1c339cecdca5 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:43.182089 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c392e-2a24-40f6-b131-7dc62e57c68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:43.183010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c392e-2a24-40f6-b131-7dc62e57c68e admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:43.183534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c392e-2a24-40f6-b131-7dc62e57c68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:43.183816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c392e-2a24-40f6-b131-7dc62e57c68e admin admin] PATH_INFO: `/v3/projects/bf062d23afb54047835a2f166a938f02/users/d22fbcce79f94e3bbfe39730b609985f/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:43.193400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2bd4c-d9d2-4ae6-b2e1-c5b6a882496c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:43.197511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a1112-30ec-4ff2-81ba-c7ae1f4f8bd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:43.198154 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a1112-30ec-4ff2-81ba-c7ae1f4f8bd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:43.198434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a1112-30ec-4ff2-81ba-c7ae1f4f8bd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:43.198823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a1112-30ec-4ff2-81ba-c7ae1f4f8bd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:43.199710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c392e-2a24-40f6-b131-7dc62e57c68e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf062d23afb54047835a2f166a938f02, user_id=d22fbcce79f94e3bbfe39730b609985f, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:43.202423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c392e-2a24-40f6-b131-7dc62e57c68e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:43.208396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a1112-30ec-4ff2-81ba-c7ae1f4f8bd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:43.209994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a1112-30ec-4ff2-81ba-c7ae1f4f8bd7 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:43.217069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1024/2053] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:43 2022] PUT /identity/v3/projects/bf062d23afb54047835a2f166a938f02/users/d22fbcce79f94e3bbfe39730b609985f/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:43.223996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1030/2054] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:43.229241 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951b132-ec59-4db3-bacb-4c1f3f132e83 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:43.230185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951b132-ec59-4db3-bacb-4c1f3f132e83 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:43.230457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951b132-ec59-4db3-bacb-4c1f3f132e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:43.260293 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-b951b132-ec59-4db3-bacb-4c1f3f132e83 None None] MFA Rules not processed for user `d22fbcce79f94e3bbfe39730b609985f`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:43.261689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b951b132-ec59-4db3-bacb-4c1f3f132e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:43.338439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1025/2055] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:43.350404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a1112-30ec-4ff2-81ba-c7ae1f4f8bd7 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:43.352059 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8b5a1112-30ec-4ff2-81ba-c7ae1f4f8bd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:43.417211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a1112-30ec-4ff2-81ba-c7ae1f4f8bd7 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:43.463736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:43.469074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cb56d-8626-4c04-bcb7-cf81d9a777ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48586ff1269f49a3ae3d9f3e40c1b9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:43.469730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cb56d-8626-4c04-bcb7-cf81d9a777ee service glance] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:43.470016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cb56d-8626-4c04-bcb7-cf81d9a777ee service glance] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:43.470291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cb56d-8626-4c04-bcb7-cf81d9a777ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:43.478663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31cb56d-8626-4c04-bcb7-cf81d9a777ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:43.480116 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31cb56d-8626-4c04-bcb7-cf81d9a777ee service glance] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:43.489143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1031/2056] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:43.520205 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951b132-ec59-4db3-bacb-4c1f3f132e83 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:43.521647 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b951b132-ec59-4db3-bacb-4c1f3f132e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:43.612481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951b132-ec59-4db3-bacb-4c1f3f132e83 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:43.617506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eecaa0-099e-4e5f-9dcb-568b038db526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:43.618462 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eecaa0-099e-4e5f-9dcb-568b038db526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:43.618744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eecaa0-099e-4e5f-9dcb-568b038db526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:43.619020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eecaa0-099e-4e5f-9dcb-568b038db526 admin admin] PATH_INFO: `/v3/projects/cabdc25e41ac45ae824567f3b2aca39c` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:43.620311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eecaa0-099e-4e5f-9dcb-568b038db526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cabdc25e41ac45ae824567f3b2aca39c)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:43.622236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eecaa0-099e-4e5f-9dcb-568b038db526 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:43.673441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f1eecaa0-099e-4e5f-9dcb-568b038db526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cabdc25e41ac45ae824567f3b2aca39c', 'request_id': 'req-f1eecaa0-099e-4e5f-9dcb-568b038db526'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:43.681142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f1eecaa0-099e-4e5f-9dcb-568b038db526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cabdc25e41ac45ae824567f3b2aca39c', 'request_id': 'req-f1eecaa0-099e-4e5f-9dcb-568b038db526'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:43.687443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f1eecaa0-099e-4e5f-9dcb-568b038db526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cabdc25e41ac45ae824567f3b2aca39c', 'request_id': 'req-f1eecaa0-099e-4e5f-9dcb-568b038db526'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:43.688698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:43.702109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f1eecaa0-099e-4e5f-9dcb-568b038db526 admin admin] The token cache is being invalidate because project cabdc25e41ac45ae824567f3b2aca39c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:43.702446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f1eecaa0-099e-4e5f-9dcb-568b038db526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:43.703140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f1eecaa0-099e-4e5f-9dcb-568b038db526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:43.704699 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1026/2057] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:43 2022] DELETE /identity/v3/projects/cabdc25e41ac45ae824567f3b2aca39c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:43.709669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af38f0-ea07-4813-9ae3-63bb219b0906 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:43.710134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af38f0-ea07-4813-9ae3-63bb219b0906 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:43.710360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af38f0-ea07-4813-9ae3-63bb219b0906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:43.729863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-99af38f0-ea07-4813-9ae3-63bb219b0906 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:43.731561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-99af38f0-ea07-4813-9ae3-63bb219b0906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:43.792234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1032/2058] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:20:43.800422 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fad491-8ad0-48bf-94a9-2081485f8620 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:43.800724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fad491-8ad0-48bf-94a9-2081485f8620 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:43.801007 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fad491-8ad0-48bf-94a9-2081485f8620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:43.824190 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-75fad491-8ad0-48bf-94a9-2081485f8620 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:43.825427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-75fad491-8ad0-48bf-94a9-2081485f8620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:43.833876 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1027/2059] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:43.839177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af38f0-ea07-4813-9ae3-63bb219b0906 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:43.851479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af38f0-ea07-4813-9ae3-63bb219b0906 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:43.857255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e242a4a-4217-4608-9249-42ebaa855e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:43.857932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e242a4a-4217-4608-9249-42ebaa855e75 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:43.858221 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e242a4a-4217-4608-9249-42ebaa855e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:43.858504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e242a4a-4217-4608-9249-42ebaa855e75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:43.859269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e242a4a-4217-4608-9249-42ebaa855e75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:43.859574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e242a4a-4217-4608-9249-42ebaa855e75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:43.861277 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e242a4a-4217-4608-9249-42ebaa855e75 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:43.873351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1033/2060] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:20:43.879671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fad491-8ad0-48bf-94a9-2081485f8620 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:43.889200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fad491-8ad0-48bf-94a9-2081485f8620 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:43.894093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a254fd-7abe-485b-b5dd-7de65ed293de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:43.894956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a254fd-7abe-485b-b5dd-7de65ed293de admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:43.895381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a254fd-7abe-485b-b5dd-7de65ed293de admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:43.895828 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a254fd-7abe-485b-b5dd-7de65ed293de admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:43.896685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a254fd-7abe-485b-b5dd-7de65ed293de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:43.899719 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a254fd-7abe-485b-b5dd-7de65ed293de admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:43.940681 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1028/2061] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:43 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:43.944460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e242a4a-4217-4608-9249-42ebaa855e75 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:43.959735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e242a4a-4217-4608-9249-42ebaa855e75 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:44.045170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16579c-3023-407a-8c49-ee426b36b9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:44.045800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16579c-3023-407a-8c49-ee426b36b9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.046090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16579c-3023-407a-8c49-ee426b36b9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.046371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16579c-3023-407a-8c49-ee426b36b9c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.047067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16579c-3023-407a-8c49-ee426b36b9c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:44.049132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16579c-3023-407a-8c49-ee426b36b9c2 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:44.075469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1034/2062] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:43 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:20:44.076686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a254fd-7abe-485b-b5dd-7de65ed293de admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:44.087542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a254fd-7abe-485b-b5dd-7de65ed293de admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:44.091915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ea29a-a3c8-4588-9e3c-b70c9c2e6571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:44.092600 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ea29a-a3c8-4588-9e3c-b70c9c2e6571 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.092883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ea29a-a3c8-4588-9e3c-b70c9c2e6571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.093160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ea29a-a3c8-4588-9e3c-b70c9c2e6571 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.093882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ea29a-a3c8-4588-9e3c-b70c9c2e6571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:44.095523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ea29a-a3c8-4588-9e3c-b70c9c2e6571 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:44.109824 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1029/2063] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:44.113748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16579c-3023-407a-8c49-ee426b36b9c2 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:44.123754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16579c-3023-407a-8c49-ee426b36b9c2 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:44.128424 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204edfcd-6588-449b-89c7-349890dd6fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:44.129084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204edfcd-6588-449b-89c7-349890dd6fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.129371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204edfcd-6588-449b-89c7-349890dd6fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.129651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204edfcd-6588-449b-89c7-349890dd6fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.130397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204edfcd-6588-449b-89c7-349890dd6fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:44.131991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204edfcd-6588-449b-89c7-349890dd6fdb admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:44.142283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1035/2064] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:44.146241 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ea29a-a3c8-4588-9e3c-b70c9c2e6571 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:44.156857 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ea29a-a3c8-4588-9e3c-b70c9c2e6571 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:44.161455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cfec9-5378-4f05-a828-49a6b9e46ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:44.162572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cfec9-5378-4f05-a828-49a6b9e46ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.162852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cfec9-5378-4f05-a828-49a6b9e46ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.163127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cfec9-5378-4f05-a828-49a6b9e46ddf admin admin] PATH_INFO: `/v3/projects/737082f191774433bbdb7d29fcc6d191/users/cb658edc76e94a208c298833b7351590/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.184191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cfec9-5378-4f05-a828-49a6b9e46ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=737082f191774433bbdb7d29fcc6d191, user_id=cb658edc76e94a208c298833b7351590, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:44.187019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cfec9-5378-4f05-a828-49a6b9e46ddf admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:44.203092 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1030/2065] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:44 2022] PUT /identity/v3/projects/737082f191774433bbdb7d29fcc6d191/users/cb658edc76e94a208c298833b7351590/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:44.215131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e8a82-1bf4-4602-867b-49b4b232a52a None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.215804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e8a82-1bf4-4602-867b-49b4b232a52a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.215804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e8a82-1bf4-4602-867b-49b4b232a52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.256433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-0b3e8a82-1bf4-4602-867b-49b4b232a52a None None] MFA Rules not processed for user `cb658edc76e94a208c298833b7351590`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:44.259049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-0b3e8a82-1bf4-4602-867b-49b4b232a52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:44.309979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:44.337782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1036/2066] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:44.339190 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cfec9-5378-4f05-a828-49a6b9e46ddf admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:44.414404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cfec9-5378-4f05-a828-49a6b9e46ddf admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:44.419105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a86bf-b167-43eb-b601-68283a2a6b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:44.419769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a86bf-b167-43eb-b601-68283a2a6b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.420054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a86bf-b167-43eb-b601-68283a2a6b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.420373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a86bf-b167-43eb-b601-68283a2a6b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.421057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a86bf-b167-43eb-b601-68283a2a6b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:44.422973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a86bf-b167-43eb-b601-68283a2a6b2a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:44.450353 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1031/2067] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:44 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:44.454741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e8a82-1bf4-4602-867b-49b4b232a52a None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:44.465372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e8a82-1bf4-4602-867b-49b4b232a52a None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:44.521726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd8659-879c-4825-bc56-508858d2f7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:44.522410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd8659-879c-4825-bc56-508858d2f7af admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.522697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd8659-879c-4825-bc56-508858d2f7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.522978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd8659-879c-4825-bc56-508858d2f7af admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.523733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd8659-879c-4825-bc56-508858d2f7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:44.525707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd8659-879c-4825-bc56-508858d2f7af admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:44.547148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1037/2068] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:44 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:44.551468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a86bf-b167-43eb-b601-68283a2a6b2a admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:44.563110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a86bf-b167-43eb-b601-68283a2a6b2a admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:44.568409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb73cf-93a6-4166-848b-6d67b6ada04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:44.569209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb73cf-93a6-4166-848b-6d67b6ada04f admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.569515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb73cf-93a6-4166-848b-6d67b6ada04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.569793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb73cf-93a6-4166-848b-6d67b6ada04f admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.570578 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb73cf-93a6-4166-848b-6d67b6ada04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:44.572843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb73cf-93a6-4166-848b-6d67b6ada04f admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:44.584662 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1032/2069] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:44.588716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd8659-879c-4825-bc56-508858d2f7af admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:44.599137 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd8659-879c-4825-bc56-508858d2f7af admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:44.604137 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5763ca-4370-498f-8e8a-472b7d97bd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:44.604805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5763ca-4370-498f-8e8a-472b7d97bd70 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.605201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5763ca-4370-498f-8e8a-472b7d97bd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.605420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5763ca-4370-498f-8e8a-472b7d97bd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.606138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5763ca-4370-498f-8e8a-472b7d97bd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:44.607814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5763ca-4370-498f-8e8a-472b7d97bd70 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:44.619201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1038/2070] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:44.623401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb73cf-93a6-4166-848b-6d67b6ada04f admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:44.634101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb73cf-93a6-4166-848b-6d67b6ada04f admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:44.638976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf948d-040d-4e2e-acd8-f2463d5d1e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:44.639883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf948d-040d-4e2e-acd8-f2463d5d1e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.640212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf948d-040d-4e2e-acd8-f2463d5d1e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.640520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf948d-040d-4e2e-acd8-f2463d5d1e12 admin admin] PATH_INFO: `/v3/projects/a8e981179cfb4bfcbb74ba1cf67b5edb/users/f48a099bc34f4939bbba81823ab10ceb/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.654723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf948d-040d-4e2e-acd8-f2463d5d1e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e981179cfb4bfcbb74ba1cf67b5edb, user_id=f48a099bc34f4939bbba81823ab10ceb, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:44.657441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf948d-040d-4e2e-acd8-f2463d5d1e12 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:44.670292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1033/2071] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:44 2022] PUT /identity/v3/projects/a8e981179cfb4bfcbb74ba1cf67b5edb/users/f48a099bc34f4939bbba81823ab10ceb/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:44.680834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f46d8-7f62-4b46-96df-afde29b729d5 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.681126 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f46d8-7f62-4b46-96df-afde29b729d5 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.681426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f46d8-7f62-4b46-96df-afde29b729d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.709971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c11f46d8-7f62-4b46-96df-afde29b729d5 None None] MFA Rules not processed for user `f48a099bc34f4939bbba81823ab10ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:44.711197 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c11f46d8-7f62-4b46-96df-afde29b729d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:44.776927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1039/2072] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:44.781511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf948d-040d-4e2e-acd8-f2463d5d1e12 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:44.821953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:44.847518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf948d-040d-4e2e-acd8-f2463d5d1e12 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:44.852840 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fed84-f304-4bc9-a315-c58b0d1d6239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:44.853484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fed84-f304-4bc9-a315-c58b0d1d6239 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.853769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fed84-f304-4bc9-a315-c58b0d1d6239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.854048 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fed84-f304-4bc9-a315-c58b0d1d6239 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.854735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125fed84-f304-4bc9-a315-c58b0d1d6239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:44.856683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125fed84-f304-4bc9-a315-c58b0d1d6239 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:44.881651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1034/2073] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:44 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:20:44.885684 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f46d8-7f62-4b46-96df-afde29b729d5 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:44.896547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f46d8-7f62-4b46-96df-afde29b729d5 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:44.947302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:44.955184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f19cf6-9f9a-4711-b841-e2ec1a88751d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:44.955862 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f19cf6-9f9a-4711-b841-e2ec1a88751d admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:44.956172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f19cf6-9f9a-4711-b841-e2ec1a88751d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:44.956449 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f19cf6-9f9a-4711-b841-e2ec1a88751d admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:44.957169 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f19cf6-9f9a-4711-b841-e2ec1a88751d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:44.959120 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f19cf6-9f9a-4711-b841-e2ec1a88751d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:44.979405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1040/2074] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:44 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:44.983569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fed84-f304-4bc9-a315-c58b0d1d6239 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:44.997128 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fed84-f304-4bc9-a315-c58b0d1d6239 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:45.003225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129c6da-e37c-496d-bbd7-eda0db97c42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:45.003961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129c6da-e37c-496d-bbd7-eda0db97c42e admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:45.004337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129c6da-e37c-496d-bbd7-eda0db97c42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:45.004680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129c6da-e37c-496d-bbd7-eda0db97c42e admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:45.005490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129c6da-e37c-496d-bbd7-eda0db97c42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:45.007152 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129c6da-e37c-496d-bbd7-eda0db97c42e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:45.017946 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1035/2075] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:45.022089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f19cf6-9f9a-4711-b841-e2ec1a88751d admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:45.032012 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f19cf6-9f9a-4711-b841-e2ec1a88751d admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:45.037586 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685650b2-6838-4544-ab0c-107b23a06afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:45.038538 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685650b2-6838-4544-ab0c-107b23a06afb admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:45.039436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685650b2-6838-4544-ab0c-107b23a06afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:45.041341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685650b2-6838-4544-ab0c-107b23a06afb admin admin] PATH_INFO: `/v3/projects/9c43e24ab5784f068c4338721bc0a7c9/users/780a3cc4683e4bc99907f0a6afbe5425/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:45.057042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685650b2-6838-4544-ab0c-107b23a06afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c43e24ab5784f068c4338721bc0a7c9, user_id=780a3cc4683e4bc99907f0a6afbe5425, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:45.059747 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685650b2-6838-4544-ab0c-107b23a06afb admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:45.074430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1041/2076] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:45 2022] PUT /identity/v3/projects/9c43e24ab5784f068c4338721bc0a7c9/users/780a3cc4683e4bc99907f0a6afbe5425/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:45.079014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129c6da-e37c-496d-bbd7-eda0db97c42e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:45.091135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129c6da-e37c-496d-bbd7-eda0db97c42e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:45.152744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7981fa-97af-4ddd-93f3-6f970d9a68da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:45.153432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7981fa-97af-4ddd-93f3-6f970d9a68da admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:45.153714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7981fa-97af-4ddd-93f3-6f970d9a68da admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:45.154016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7981fa-97af-4ddd-93f3-6f970d9a68da admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:45.154728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7981fa-97af-4ddd-93f3-6f970d9a68da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:45.156403 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7981fa-97af-4ddd-93f3-6f970d9a68da admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:45.167267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1036/2077] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:20:45.171452 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685650b2-6838-4544-ab0c-107b23a06afb admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:45.182049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685650b2-6838-4544-ab0c-107b23a06afb admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:45.186604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85c724-aa57-4511-a8e0-81f7eecea89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:45.187478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85c724-aa57-4511-a8e0-81f7eecea89f admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:45.187763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85c724-aa57-4511-a8e0-81f7eecea89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:45.188038 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85c724-aa57-4511-a8e0-81f7eecea89f admin admin] PATH_INFO: `/v3/projects/9c43e24ab5784f068c4338721bc0a7c9/users/780a3cc4683e4bc99907f0a6afbe5425/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:45.190058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85c724-aa57-4511-a8e0-81f7eecea89f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c43e24ab5784f068c4338721bc0a7c9, user_id=780a3cc4683e4bc99907f0a6afbe5425, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:45.192742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85c724-aa57-4511-a8e0-81f7eecea89f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:45.208010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1042/2078] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:45 2022] PUT /identity/v3/projects/9c43e24ab5784f068c4338721bc0a7c9/users/780a3cc4683e4bc99907f0a6afbe5425/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:45.217119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa42e-1993-4b9f-b169-9d99ff81831a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:45.217412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa42e-1993-4b9f-b169-9d99ff81831a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:45.217693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa42e-1993-4b9f-b169-9d99ff81831a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:45.247772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-1c1fa42e-1993-4b9f-b169-9d99ff81831a None None] MFA Rules not processed for user `780a3cc4683e4bc99907f0a6afbe5425`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:45.249109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-1c1fa42e-1993-4b9f-b169-9d99ff81831a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:45.328149 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1037/2079] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:45.341202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85c724-aa57-4511-a8e0-81f7eecea89f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:45.342814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1c85c724-aa57-4511-a8e0-81f7eecea89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:45.402215 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85c724-aa57-4511-a8e0-81f7eecea89f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:45.405882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bfbb1-f988-48d0-86de-47015c5b7cdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:45.406525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bfbb1-f988-48d0-86de-47015c5b7cdc service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:45.406811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bfbb1-f988-48d0-86de-47015c5b7cdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:45.407087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bfbb1-f988-48d0-86de-47015c5b7cdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:45.465296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bfbb1-f988-48d0-86de-47015c5b7cdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:45.466745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bfbb1-f988-48d0-86de-47015c5b7cdc service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:45.475563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1043/2080] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:46.958430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa42e-1993-4b9f-b169-9d99ff81831a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:46.968634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa42e-1993-4b9f-b169-9d99ff81831a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:46.972154 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb16129-a4c7-4d91-8701-f0bc3a688904 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:46.972810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb16129-a4c7-4d91-8701-f0bc3a688904 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:46.973069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb16129-a4c7-4d91-8701-f0bc3a688904 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:46.973365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb16129-a4c7-4d91-8701-f0bc3a688904 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:47.001968 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:47.033273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb16129-a4c7-4d91-8701-f0bc3a688904 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:47.034626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb16129-a4c7-4d91-8701-f0bc3a688904 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:47.044758 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1038/2081] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:47.606748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bfbb1-f988-48d0-86de-47015c5b7cdc service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:47.615543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bfbb1-f988-48d0-86de-47015c5b7cdc service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:47.620329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3e1ee-5d75-42a6-b021-978ba92163dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:47.620997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3e1ee-5d75-42a6-b021-978ba92163dd service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:47.621288 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3e1ee-5d75-42a6-b021-978ba92163dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:47.621601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3e1ee-5d75-42a6-b021-978ba92163dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:47.631442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3e1ee-5d75-42a6-b021-978ba92163dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:47.632907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3e1ee-5d75-42a6-b021-978ba92163dd service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:47.643132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1044/2082] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:51.485858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb16129-a4c7-4d91-8701-f0bc3a688904 service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:51.551659 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb16129-a4c7-4d91-8701-f0bc3a688904 service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:51.556352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:51.557898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:51.558198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:51.558480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] PATH_INFO: `/v3/users/cb658edc76e94a208c298833b7351590` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:51.559685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb658edc76e94a208c298833b7351590)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:51.561880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:51.609445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb658edc76e94a208c298833b7351590'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:51.610108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb658edc76e94a208c298833b7351590'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:51.617110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb658edc76e94a208c298833b7351590'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:51.629613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb658edc76e94a208c298833b7351590'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:51.630328 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb658edc76e94a208c298833b7351590'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:51.644334 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1039/2083] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:51 2022] DELETE /identity/v3/users/cb658edc76e94a208c298833b7351590 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:51.648807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab93a46-ce3b-4467-bce5-257b50a995aa None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:51.649369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab93a46-ce3b-4467-bce5-257b50a995aa None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:51.649460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab93a46-ce3b-4467-bce5-257b50a995aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:51.668993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-5ab93a46-ce3b-4467-bce5-257b50a995aa None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:51.670222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5ab93a46-ce3b-4467-bce5-257b50a995aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:51.713162 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:51.731269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1045/2084] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:51.740828 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:51.742226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:51.797906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:51.802882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f494fc3-b24b-496f-8120-daae6261118c admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:51.807481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d73f42-bd35-42e7-a4b8-f663371e7331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:51.808175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d73f42-bd35-42e7-a4b8-f663371e7331 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:51.808873 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d73f42-bd35-42e7-a4b8-f663371e7331 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:51.809238 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d73f42-bd35-42e7-a4b8-f663371e7331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:51.819510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d73f42-bd35-42e7-a4b8-f663371e7331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:51.821022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d73f42-bd35-42e7-a4b8-f663371e7331 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:51.830481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1040/2085] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:52.242985 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab93a46-ce3b-4467-bce5-257b50a995aa None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:52.245271 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5ab93a46-ce3b-4467-bce5-257b50a995aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:52.257246 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab93a46-ce3b-4467-bce5-257b50a995aa None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:52.262283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:52.263057 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:52.263423 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:52.263810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] PATH_INFO: `/v3/projects/737082f191774433bbdb7d29fcc6d191` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:52.269073 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] RBAC: Authorizing `identity:delete_project(project_id=737082f191774433bbdb7d29fcc6d191)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:52.272603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:52.336730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '737082f191774433bbdb7d29fcc6d191', 'request_id': 'req-27c57882-df19-4cb9-8954-387fdb25fedd'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:52.345907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '737082f191774433bbdb7d29fcc6d191', 'request_id': 'req-27c57882-df19-4cb9-8954-387fdb25fedd'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:52.354705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '737082f191774433bbdb7d29fcc6d191', 'request_id': 'req-27c57882-df19-4cb9-8954-387fdb25fedd'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:52.372052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] The token cache is being invalidate because project 737082f191774433bbdb7d29fcc6d191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:52.372465 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:52.373152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:52.375064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1046/2086] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:52 2022] DELETE /identity/v3/projects/737082f191774433bbdb7d29fcc6d191 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:52.379044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d73f42-bd35-42e7-a4b8-f663371e7331 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:52.389855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d73f42-bd35-42e7-a4b8-f663371e7331 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:52.465638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:52.466522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:52.466811 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:52.467091 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] PATH_INFO: `/v3/users/f48a099bc34f4939bbba81823ab10ceb` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:52.468322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f48a099bc34f4939bbba81823ab10ceb)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:52.470267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:52.521439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f48a099bc34f4939bbba81823ab10ceb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:52.522516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f48a099bc34f4939bbba81823ab10ceb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:52.530652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f48a099bc34f4939bbba81823ab10ceb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:52.553053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f48a099bc34f4939bbba81823ab10ceb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:52.556191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f48a099bc34f4939bbba81823ab10ceb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:52.570624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1041/2087] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:52 2022] DELETE /identity/v3/users/f48a099bc34f4939bbba81823ab10ceb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:52.894894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:52.897266 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:52.937722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c57882-df19-4cb9-8954-387fdb25fedd admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:53.017318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:53.018267 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:53.018579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:53.018888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] PATH_INFO: `/v3/projects/a8e981179cfb4bfcbb74ba1cf67b5edb` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:53.020462 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8e981179cfb4bfcbb74ba1cf67b5edb)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:53.022575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:53.074845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8e981179cfb4bfcbb74ba1cf67b5edb', 'request_id': 'req-90581d38-597e-4539-9605-4b186c3ea873'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.081518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8e981179cfb4bfcbb74ba1cf67b5edb', 'request_id': 'req-90581d38-597e-4539-9605-4b186c3ea873'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.089517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8e981179cfb4bfcbb74ba1cf67b5edb', 'request_id': 'req-90581d38-597e-4539-9605-4b186c3ea873'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.105099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] The token cache is being invalidate because project a8e981179cfb4bfcbb74ba1cf67b5edb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:53.105293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.105986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.107578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1047/2088] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:52 2022] DELETE /identity/v3/projects/a8e981179cfb4bfcbb74ba1cf67b5edb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:53.111924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:53.113417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:53.127400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bee9b-c6d0-4fd5-b8d1-ea2046c83532 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:53.221284 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7d3cf-95ce-4797-8cd1-c750f952be01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:53.222975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7d3cf-95ce-4797-8cd1-c750f952be01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:53.223736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7d3cf-95ce-4797-8cd1-c750f952be01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:53.224242 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7d3cf-95ce-4797-8cd1-c750f952be01 admin admin] PATH_INFO: `/v3/users/780a3cc4683e4bc99907f0a6afbe5425` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:53.225962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7d3cf-95ce-4797-8cd1-c750f952be01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=780a3cc4683e4bc99907f0a6afbe5425)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:53.228049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7d3cf-95ce-4797-8cd1-c750f952be01 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:53.243496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:53.287391 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-09b7d3cf-95ce-4797-8cd1-c750f952be01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '780a3cc4683e4bc99907f0a6afbe5425'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.288175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-09b7d3cf-95ce-4797-8cd1-c750f952be01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '780a3cc4683e4bc99907f0a6afbe5425'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.301063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-09b7d3cf-95ce-4797-8cd1-c750f952be01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '780a3cc4683e4bc99907f0a6afbe5425'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.313980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-09b7d3cf-95ce-4797-8cd1-c750f952be01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '780a3cc4683e4bc99907f0a6afbe5425'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.314728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-09b7d3cf-95ce-4797-8cd1-c750f952be01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '780a3cc4683e4bc99907f0a6afbe5425'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.330953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1042/2089] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:53 2022] DELETE /identity/v3/users/780a3cc4683e4bc99907f0a6afbe5425 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:53.624659 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:53.626125 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:53.634045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:53.640232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90581d38-597e-4539-9605-4b186c3ea873 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:53.705668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe551e-d790-4376-9a1b-b7c66bb29a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:53.706554 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe551e-d790-4376-9a1b-b7c66bb29a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:53.706958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe551e-d790-4376-9a1b-b7c66bb29a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:53.707156 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe551e-d790-4376-9a1b-b7c66bb29a30 admin admin] PATH_INFO: `/v3/projects/9c43e24ab5784f068c4338721bc0a7c9` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:53.708664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe551e-d790-4376-9a1b-b7c66bb29a30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c43e24ab5784f068c4338721bc0a7c9)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:53.710646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe551e-d790-4376-9a1b-b7c66bb29a30 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:53.760879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1cbe551e-d790-4376-9a1b-b7c66bb29a30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c43e24ab5784f068c4338721bc0a7c9', 'request_id': 'req-1cbe551e-d790-4376-9a1b-b7c66bb29a30'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.766791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1cbe551e-d790-4376-9a1b-b7c66bb29a30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c43e24ab5784f068c4338721bc0a7c9', 'request_id': 'req-1cbe551e-d790-4376-9a1b-b7c66bb29a30'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.773039 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1cbe551e-d790-4376-9a1b-b7c66bb29a30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c43e24ab5784f068c4338721bc0a7c9', 'request_id': 'req-1cbe551e-d790-4376-9a1b-b7c66bb29a30'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.785702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1cbe551e-d790-4376-9a1b-b7c66bb29a30 admin admin] The token cache is being invalidate because project 9c43e24ab5784f068c4338721bc0a7c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:53.785995 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1cbe551e-d790-4376-9a1b-b7c66bb29a30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.786966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1cbe551e-d790-4376-9a1b-b7c66bb29a30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:53.788689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1048/2090] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:53 2022] DELETE /identity/v3/projects/9c43e24ab5784f068c4338721bc0a7c9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:53.794134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a7191-4b4e-48eb-b656-c75c3ab66588 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:53.794645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a7191-4b4e-48eb-b656-c75c3ab66588 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:53.794739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a7191-4b4e-48eb-b656-c75c3ab66588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:53.818106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-3e3a7191-4b4e-48eb-b656-c75c3ab66588 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:53.819417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3e3a7191-4b4e-48eb-b656-c75c3ab66588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:53.886580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1043/2091] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:20:53.894627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb084f8-a76b-4acb-8302-b4dad253b26b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:53.894921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb084f8-a76b-4acb-8302-b4dad253b26b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:53.895512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb084f8-a76b-4acb-8302-b4dad253b26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:53.916845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-dcb084f8-a76b-4acb-8302-b4dad253b26b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:53.917977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-dcb084f8-a76b-4acb-8302-b4dad253b26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:53.925825 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1049/2092] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:53.930105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a7191-4b4e-48eb-b656-c75c3ab66588 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:53.939893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a7191-4b4e-48eb-b656-c75c3ab66588 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:53.944559 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dfb10-a7e0-41b3-b839-ef4034009202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:53.945238 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dfb10-a7e0-41b3-b839-ef4034009202 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:53.945528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dfb10-a7e0-41b3-b839-ef4034009202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:53.945851 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dfb10-a7e0-41b3-b839-ef4034009202 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:53.946583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363dfb10-a7e0-41b3-b839-ef4034009202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:20:53.946895 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363dfb10-a7e0-41b3-b839-ef4034009202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:53.949066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363dfb10-a7e0-41b3-b839-ef4034009202 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:53.961473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1044/2093] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:20:53.966208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb084f8-a76b-4acb-8302-b4dad253b26b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:53.975685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb084f8-a76b-4acb-8302-b4dad253b26b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:53.980556 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a97e7-8b28-4160-af5c-7f29196456a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:53.981440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a97e7-8b28-4160-af5c-7f29196456a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:53.981726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a97e7-8b28-4160-af5c-7f29196456a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:53.982145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a97e7-8b28-4160-af5c-7f29196456a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:53.982882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a97e7-8b28-4160-af5c-7f29196456a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:53.985092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a97e7-8b28-4160-af5c-7f29196456a2 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:54.013765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1050/2094] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:20:53 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:20:54.017913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dfb10-a7e0-41b3-b839-ef4034009202 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:54.027956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dfb10-a7e0-41b3-b839-ef4034009202 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:54.085075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b050ae-7054-4da8-99e1-002545553380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:54.085740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b050ae-7054-4da8-99e1-002545553380 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:54.086029 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b050ae-7054-4da8-99e1-002545553380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:54.086311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b050ae-7054-4da8-99e1-002545553380 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:54.086999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b050ae-7054-4da8-99e1-002545553380 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:54.088960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b050ae-7054-4da8-99e1-002545553380 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:54.108150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1045/2095] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:20:54 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:20:54.112262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a97e7-8b28-4160-af5c-7f29196456a2 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:54.125838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a97e7-8b28-4160-af5c-7f29196456a2 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:54.130582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201470c-5480-496d-a57c-90eed8ca2c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:54.131254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201470c-5480-496d-a57c-90eed8ca2c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:54.131539 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201470c-5480-496d-a57c-90eed8ca2c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:54.131817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201470c-5480-496d-a57c-90eed8ca2c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:54.132576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201470c-5480-496d-a57c-90eed8ca2c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:54.134181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201470c-5480-496d-a57c-90eed8ca2c61 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:54.145246 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1051/2096] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:54.149427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b050ae-7054-4da8-99e1-002545553380 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:54.159381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b050ae-7054-4da8-99e1-002545553380 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:54.164341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fc6a2-5478-4b07-a84e-b3c20e7de5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:54.164998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fc6a2-5478-4b07-a84e-b3c20e7de5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:54.165288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fc6a2-5478-4b07-a84e-b3c20e7de5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:54.165616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fc6a2-5478-4b07-a84e-b3c20e7de5c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:54.166413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fc6a2-5478-4b07-a84e-b3c20e7de5c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:54.168097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fc6a2-5478-4b07-a84e-b3c20e7de5c5 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:54.169324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:54.179621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1046/2097] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:20:54.183973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201470c-5480-496d-a57c-90eed8ca2c61 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:54.193890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201470c-5480-496d-a57c-90eed8ca2c61 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:54.199004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad7d49-9185-41c9-9c0d-9f1685bfda8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:54.199871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad7d49-9185-41c9-9c0d-9f1685bfda8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:54.200192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad7d49-9185-41c9-9c0d-9f1685bfda8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:54.200475 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad7d49-9185-41c9-9c0d-9f1685bfda8b admin admin] PATH_INFO: `/v3/projects/656ac358f6f44d7180b1122e87dc9c3e/users/9710f6fb31da4df39b04839a29dfdb34/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:54.216574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad7d49-9185-41c9-9c0d-9f1685bfda8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=656ac358f6f44d7180b1122e87dc9c3e, user_id=9710f6fb31da4df39b04839a29dfdb34, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:54.219334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad7d49-9185-41c9-9c0d-9f1685bfda8b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:54.233458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1052/2098] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:20:54 2022] PUT /identity/v3/projects/656ac358f6f44d7180b1122e87dc9c3e/users/9710f6fb31da4df39b04839a29dfdb34/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:54.239229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b23383-69da-4c5b-b066-72d0b0ebe665 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:54.239563 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b23383-69da-4c5b-b066-72d0b0ebe665 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:54.239849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b23383-69da-4c5b-b066-72d0b0ebe665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:54.257976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-13b23383-69da-4c5b-b066-72d0b0ebe665 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:54.259184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-13b23383-69da-4c5b-b066-72d0b0ebe665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:54.318363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1047/2099] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:54.329123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad7d49-9185-41c9-9c0d-9f1685bfda8b admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:54.333178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b3ad7d49-9185-41c9-9c0d-9f1685bfda8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:54.366531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:54.405315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad7d49-9185-41c9-9c0d-9f1685bfda8b admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:54.409049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98957af0-5f11-4a4f-be7d-093f35643b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:54.409797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98957af0-5f11-4a4f-be7d-093f35643b18 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:54.410102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98957af0-5f11-4a4f-be7d-093f35643b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:54.410368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98957af0-5f11-4a4f-be7d-093f35643b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:54.420797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98957af0-5f11-4a4f-be7d-093f35643b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:54.422199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98957af0-5f11-4a4f-be7d-093f35643b18 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:54.437444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1053/2100] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:55.127935 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365ebe2-c3b6-4584-a88e-2812fd455fa1 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:55.129157 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365ebe2-c3b6-4584-a88e-2812fd455fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:55.129452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365ebe2-c3b6-4584-a88e-2812fd455fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:55.155428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-5365ebe2-c3b6-4584-a88e-2812fd455fa1 None None] MFA Rules not processed for user `9710f6fb31da4df39b04839a29dfdb34`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:55.156669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-5365ebe2-c3b6-4584-a88e-2812fd455fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:55.233031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1048/2101] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:55.246585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98957af0-5f11-4a4f-be7d-093f35643b18 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:55.248040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-98957af0-5f11-4a4f-be7d-093f35643b18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:55.310976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98957af0-5f11-4a4f-be7d-093f35643b18 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:55.315712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4449a-28bf-4c32-84d4-b1dbf421d3da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:55.316473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4449a-28bf-4c32-84d4-b1dbf421d3da service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:55.316765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4449a-28bf-4c32-84d4-b1dbf421d3da service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:55.317049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4449a-28bf-4c32-84d4-b1dbf421d3da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:55.325971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b4449a-28bf-4c32-84d4-b1dbf421d3da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:55.327446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b4449a-28bf-4c32-84d4-b1dbf421d3da service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:55.336359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1054/2102] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:56.695144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d73947-5030-475f-bc66-b786c321f0ab None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:56.695635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d73947-5030-475f-bc66-b786c321f0ab None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:56.695726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d73947-5030-475f-bc66-b786c321f0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:56.713667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-21d73947-5030-475f-bc66-b786c321f0ab None None] MFA Rules not processed for user `9710f6fb31da4df39b04839a29dfdb34`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:56.715423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-21d73947-5030-475f-bc66-b786c321f0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:56.722600 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1049/2103] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:59.195165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7ee4e-e8ab-4da0-881c-ccf9505b73e0 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:59.195641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7ee4e-e8ab-4da0-881c-ccf9505b73e0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:59.195926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7ee4e-e8ab-4da0-881c-ccf9505b73e0 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:59.202572 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1055/2104] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:20:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:20:59.208356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0ec0b-5668-4aa8-93bd-c0e3f1d4ad76 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:59.208887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0ec0b-5668-4aa8-93bd-c0e3f1d4ad76 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:59.209006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0ec0b-5668-4aa8-93bd-c0e3f1d4ad76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:59.230043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-92c0ec0b-5668-4aa8-93bd-c0e3f1d4ad76 None None] MFA Rules not processed for user `b8cf02a52d354517b363c0aaf477550c`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:59.231332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-92c0ec0b-5668-4aa8-93bd-c0e3f1d4ad76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:59.277198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7ee4e-e8ab-4da0-881c-ccf9505b73e0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:59.278928 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-eca7ee4e-e8ab-4da0-881c-ccf9505b73e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:59.302174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7ee4e-e8ab-4da0-881c-ccf9505b73e0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:59.307244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096b8eb-e1ce-4dcf-9694-20fc32c2b62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:59.308785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096b8eb-e1ce-4dcf-9694-20fc32c2b62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:59.309169 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096b8eb-e1ce-4dcf-9694-20fc32c2b62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:59.309520 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096b8eb-e1ce-4dcf-9694-20fc32c2b62f admin admin] PATH_INFO: `/v3/users/6d04750a83fb465e8fba24c1143d4eff` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:59.311139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096b8eb-e1ce-4dcf-9694-20fc32c2b62f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d04750a83fb465e8fba24c1143d4eff)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:59.313366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096b8eb-e1ce-4dcf-9694-20fc32c2b62f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:59.345220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1050/2105] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:20:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:20:59.372429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0ec0b-5668-4aa8-93bd-c0e3f1d4ad76 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:59.374216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-92c0ec0b-5668-4aa8-93bd-c0e3f1d4ad76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:20:59.378843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6096b8eb-e1ce-4dcf-9694-20fc32c2b62f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d04750a83fb465e8fba24c1143d4eff'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:59.395258 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0ec0b-5668-4aa8-93bd-c0e3f1d4ad76 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:59.401845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6096b8eb-e1ce-4dcf-9694-20fc32c2b62f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d04750a83fb465e8fba24c1143d4eff'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:59.402501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6096b8eb-e1ce-4dcf-9694-20fc32c2b62f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d04750a83fb465e8fba24c1143d4eff'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:59.403642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2065819-df7d-4d57-8368-08a107f971e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8cf02a52d354517b363c0aaf477550c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:59.404516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2065819-df7d-4d57-8368-08a107f971e6 service placement] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:59.404969 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2065819-df7d-4d57-8368-08a107f971e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:59.405450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2065819-df7d-4d57-8368-08a107f971e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:59.415489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2065819-df7d-4d57-8368-08a107f971e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:59.417262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2065819-df7d-4d57-8368-08a107f971e6 service placement] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:59.418124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6096b8eb-e1ce-4dcf-9694-20fc32c2b62f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d04750a83fb465e8fba24c1143d4eff'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:59.419013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6096b8eb-e1ce-4dcf-9694-20fc32c2b62f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d04750a83fb465e8fba24c1143d4eff'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:59.427690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1051/2106] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:20:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:20:59.439460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1056/2107] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:20:59 2022] DELETE /identity/v3/users/6d04750a83fb465e8fba24c1143d4eff => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:20:59.751317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2065819-df7d-4d57-8368-08a107f971e6 service placement] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:20:59.757057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:20:59.763965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2065819-df7d-4d57-8368-08a107f971e6 service placement] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:20:59.836889 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4992c6-4dc7-4d33-8626-72398d037454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:20:59.839554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4992c6-4dc7-4d33-8626-72398d037454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:59.839843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4992c6-4dc7-4d33-8626-72398d037454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:59.840197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4992c6-4dc7-4d33-8626-72398d037454 admin admin] PATH_INFO: `/v3/projects/09dfc5c0d3f4405a901acd8eeef080b9` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:59.846826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4992c6-4dc7-4d33-8626-72398d037454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09dfc5c0d3f4405a901acd8eeef080b9)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:20:59.853027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4992c6-4dc7-4d33-8626-72398d037454 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:20:59.924291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ae4992c6-4dc7-4d33-8626-72398d037454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09dfc5c0d3f4405a901acd8eeef080b9', 'request_id': 'req-ae4992c6-4dc7-4d33-8626-72398d037454'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:59.937876 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ae4992c6-4dc7-4d33-8626-72398d037454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09dfc5c0d3f4405a901acd8eeef080b9', 'request_id': 'req-ae4992c6-4dc7-4d33-8626-72398d037454'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:59.949687 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ae4992c6-4dc7-4d33-8626-72398d037454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09dfc5c0d3f4405a901acd8eeef080b9', 'request_id': 'req-ae4992c6-4dc7-4d33-8626-72398d037454'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:59.958946 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ae4992c6-4dc7-4d33-8626-72398d037454 admin admin] The token cache is being invalidate because project 09dfc5c0d3f4405a901acd8eeef080b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:20:59.959299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ae4992c6-4dc7-4d33-8626-72398d037454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:59.960014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ae4992c6-4dc7-4d33-8626-72398d037454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:20:59.961838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1052/2108] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:20:59 2022] DELETE /identity/v3/projects/09dfc5c0d3f4405a901acd8eeef080b9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:20:59.967372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b39e12-478b-4687-bb4d-f60a756ee7a3 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:20:59.967725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b39e12-478b-4687-bb4d-f60a756ee7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:20:59.968093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b39e12-478b-4687-bb4d-f60a756ee7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:20:59.986024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-49b39e12-478b-4687-bb4d-f60a756ee7a3 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:20:59.987298 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-49b39e12-478b-4687-bb4d-f60a756ee7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:00.046231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:00.058257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1057/2109] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:00.065986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29665b3-2470-4da8-a00c-d70e751450e4 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.066274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29665b3-2470-4da8-a00c-d70e751450e4 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.066554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29665b3-2470-4da8-a00c-d70e751450e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.086677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f29665b3-2470-4da8-a00c-d70e751450e4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:00.088029 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f29665b3-2470-4da8-a00c-d70e751450e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:00.095899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1053/2110] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:00.100385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b39e12-478b-4687-bb4d-f60a756ee7a3 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:00.110348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b39e12-478b-4687-bb4d-f60a756ee7a3 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:00.120717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e94cbe-8aa1-4905-8968-c633e158b72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:00.121442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e94cbe-8aa1-4905-8968-c633e158b72f admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.121739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e94cbe-8aa1-4905-8968-c633e158b72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.122018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e94cbe-8aa1-4905-8968-c633e158b72f admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.122758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e94cbe-8aa1-4905-8968-c633e158b72f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:21:00.123070 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e94cbe-8aa1-4905-8968-c633e158b72f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:00.124783 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e94cbe-8aa1-4905-8968-c633e158b72f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:00.137701 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1058/2111] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:21:00.141971 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29665b3-2470-4da8-a00c-d70e751450e4 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:00.151382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29665b3-2470-4da8-a00c-d70e751450e4 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:00.156975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3944e-1182-4e16-a5c0-122fd572ba35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:00.157830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3944e-1182-4e16-a5c0-122fd572ba35 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.158119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3944e-1182-4e16-a5c0-122fd572ba35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.158444 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3944e-1182-4e16-a5c0-122fd572ba35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.159110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3944e-1182-4e16-a5c0-122fd572ba35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:00.161082 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3944e-1182-4e16-a5c0-122fd572ba35 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:00.189518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1054/2112] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:00 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:00.194297 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e94cbe-8aa1-4905-8968-c633e158b72f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:00.206051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e94cbe-8aa1-4905-8968-c633e158b72f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:00.270040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda8df2-b33b-4f25-b421-d8fafc331281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:00.270751 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda8df2-b33b-4f25-b421-d8fafc331281 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.271079 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda8df2-b33b-4f25-b421-d8fafc331281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.271364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda8df2-b33b-4f25-b421-d8fafc331281 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.272046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda8df2-b33b-4f25-b421-d8fafc331281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:00.274035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda8df2-b33b-4f25-b421-d8fafc331281 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:00.299717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1059/2113] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:00 2022] POST /identity/v3/users => generated 279 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:00.300966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3944e-1182-4e16-a5c0-122fd572ba35 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:00.311593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3944e-1182-4e16-a5c0-122fd572ba35 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:00.316806 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fe585-f1b7-46a4-87c4-46770a7a9614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:00.317342 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fe585-f1b7-46a4-87c4-46770a7a9614 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.317682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fe585-f1b7-46a4-87c4-46770a7a9614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.317969 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fe585-f1b7-46a4-87c4-46770a7a9614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.318816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09fe585-f1b7-46a4-87c4-46770a7a9614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:00.320510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09fe585-f1b7-46a4-87c4-46770a7a9614 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:00.336174 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1055/2114] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:00.337421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda8df2-b33b-4f25-b421-d8fafc331281 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:00.347998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda8df2-b33b-4f25-b421-d8fafc331281 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:00.352855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78667a96-52d1-405f-8d1b-113a5c6cabf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:00.353539 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78667a96-52d1-405f-8d1b-113a5c6cabf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.353824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78667a96-52d1-405f-8d1b-113a5c6cabf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.354105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78667a96-52d1-405f-8d1b-113a5c6cabf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.355374 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78667a96-52d1-405f-8d1b-113a5c6cabf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:00.357143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78667a96-52d1-405f-8d1b-113a5c6cabf3 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:00.371346 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1060/2115] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:00.377884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fe585-f1b7-46a4-87c4-46770a7a9614 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:00.403763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fe585-f1b7-46a4-87c4-46770a7a9614 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:00.409898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cf6a1-2004-4e0b-bb02-28df1135cae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:00.411023 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cf6a1-2004-4e0b-bb02-28df1135cae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.411316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cf6a1-2004-4e0b-bb02-28df1135cae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.411598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cf6a1-2004-4e0b-bb02-28df1135cae2 admin admin] PATH_INFO: `/v3/projects/4eae22c650b74448ba9e8fc15e006a85/users/cebee58fba6e4efbafef8a179ed45cb8/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.431995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cf6a1-2004-4e0b-bb02-28df1135cae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eae22c650b74448ba9e8fc15e006a85, user_id=cebee58fba6e4efbafef8a179ed45cb8, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:00.435867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cf6a1-2004-4e0b-bb02-28df1135cae2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:00.455058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1056/2116] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:00 2022] PUT /identity/v3/projects/4eae22c650b74448ba9e8fc15e006a85/users/cebee58fba6e4efbafef8a179ed45cb8/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:00.465408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e041e35-9729-4343-903a-6cb723fa3a03 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.465408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e041e35-9729-4343-903a-6cb723fa3a03 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.465677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e041e35-9729-4343-903a-6cb723fa3a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.497914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-5e041e35-9729-4343-903a-6cb723fa3a03 None None] MFA Rules not processed for user `cebee58fba6e4efbafef8a179ed45cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:00.499161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5e041e35-9729-4343-903a-6cb723fa3a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:00.571637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1061/2117] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:00 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:00.576138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cf6a1-2004-4e0b-bb02-28df1135cae2 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:00.609336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:00.643693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cf6a1-2004-4e0b-bb02-28df1135cae2 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:00.648521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fd43a-ecc6-4250-a94b-c322850b2791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:00.649178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fd43a-ecc6-4250-a94b-c322850b2791 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.649458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fd43a-ecc6-4250-a94b-c322850b2791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.649731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fd43a-ecc6-4250-a94b-c322850b2791 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.650424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4fd43a-ecc6-4250-a94b-c322850b2791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:00.653003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4fd43a-ecc6-4250-a94b-c322850b2791 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:00.678933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1057/2118] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:00 2022] POST /identity/v3/projects => generated 357 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:00.682914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e041e35-9729-4343-903a-6cb723fa3a03 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:00.693347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e041e35-9729-4343-903a-6cb723fa3a03 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:00.741471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:00.758747 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db128af1-fd70-41fa-9ddc-8fefd7b08edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:00.759414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db128af1-fd70-41fa-9ddc-8fefd7b08edf admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.759699 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db128af1-fd70-41fa-9ddc-8fefd7b08edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.759977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db128af1-fd70-41fa-9ddc-8fefd7b08edf admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.761265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db128af1-fd70-41fa-9ddc-8fefd7b08edf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:00.763189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db128af1-fd70-41fa-9ddc-8fefd7b08edf admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:00.790001 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1062/2119] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:00 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:00.793813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fd43a-ecc6-4250-a94b-c322850b2791 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:00.804010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fd43a-ecc6-4250-a94b-c322850b2791 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:00.809105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e06ced-14be-48e6-b41c-66bd8eba0766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:00.809745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06ced-14be-48e6-b41c-66bd8eba0766 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.810030 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06ced-14be-48e6-b41c-66bd8eba0766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.810305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06ced-14be-48e6-b41c-66bd8eba0766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.811023 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e06ced-14be-48e6-b41c-66bd8eba0766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:00.812639 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e06ced-14be-48e6-b41c-66bd8eba0766 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:00.823021 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1058/2120] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:00.827401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db128af1-fd70-41fa-9ddc-8fefd7b08edf admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:00.837994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db128af1-fd70-41fa-9ddc-8fefd7b08edf admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:00.842794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7c5bc-92c0-469d-8929-78f58eec60c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:00.843688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7c5bc-92c0-469d-8929-78f58eec60c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.843977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7c5bc-92c0-469d-8929-78f58eec60c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.844295 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7c5bc-92c0-469d-8929-78f58eec60c1 admin admin] PATH_INFO: `/v3/projects/fecce91f10f64b8989bdf657ed870a3e/users/1741fd4b714c43b793d023b010519d6b/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.858826 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7c5bc-92c0-469d-8929-78f58eec60c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fecce91f10f64b8989bdf657ed870a3e, user_id=1741fd4b714c43b793d023b010519d6b, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:00.861518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7c5bc-92c0-469d-8929-78f58eec60c1 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:00.874702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1063/2121] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:00 2022] PUT /identity/v3/projects/fecce91f10f64b8989bdf657ed870a3e/users/1741fd4b714c43b793d023b010519d6b/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:00.878362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e06ced-14be-48e6-b41c-66bd8eba0766 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:00.889206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e06ced-14be-48e6-b41c-66bd8eba0766 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:00.953664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2177d4-a815-46bd-97fa-5a6fb86ebc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:00.954404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2177d4-a815-46bd-97fa-5a6fb86ebc79 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.954693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2177d4-a815-46bd-97fa-5a6fb86ebc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.955033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2177d4-a815-46bd-97fa-5a6fb86ebc79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.955786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2177d4-a815-46bd-97fa-5a6fb86ebc79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:00.957489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2177d4-a815-46bd-97fa-5a6fb86ebc79 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:00.971551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1059/2122] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:00.975463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7c5bc-92c0-469d-8929-78f58eec60c1 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:00.986025 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7c5bc-92c0-469d-8929-78f58eec60c1 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:00.991135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b98910-b25b-49f8-927a-b7fd85ef434e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:00.992105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b98910-b25b-49f8-927a-b7fd85ef434e admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:00.992354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b98910-b25b-49f8-927a-b7fd85ef434e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:00.992638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b98910-b25b-49f8-927a-b7fd85ef434e admin admin] PATH_INFO: `/v3/projects/fecce91f10f64b8989bdf657ed870a3e/users/1741fd4b714c43b793d023b010519d6b/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:00.994771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b98910-b25b-49f8-927a-b7fd85ef434e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fecce91f10f64b8989bdf657ed870a3e, user_id=1741fd4b714c43b793d023b010519d6b, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:00.998021 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b98910-b25b-49f8-927a-b7fd85ef434e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:01.009390 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1064/2123] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:00 2022] PUT /identity/v3/projects/fecce91f10f64b8989bdf657ed870a3e/users/1741fd4b714c43b793d023b010519d6b/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:01.020991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f2bb7-2171-4d1c-ae8b-bca5fd889fb6 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:01.021289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f2bb7-2171-4d1c-ae8b-bca5fd889fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:01.021578 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f2bb7-2171-4d1c-ae8b-bca5fd889fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:01.061934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-d25f2bb7-2171-4d1c-ae8b-bca5fd889fb6 None None] MFA Rules not processed for user `1741fd4b714c43b793d023b010519d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:01.063485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d25f2bb7-2171-4d1c-ae8b-bca5fd889fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:01.158304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1060/2124] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:01 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:01.169196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b98910-b25b-49f8-927a-b7fd85ef434e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:01.170987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d4b98910-b25b-49f8-927a-b7fd85ef434e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:01.238528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b98910-b25b-49f8-927a-b7fd85ef434e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:01.243003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff73ad5-66aa-48ba-8678-10c12259bd43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:01.243612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff73ad5-66aa-48ba-8678-10c12259bd43 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:01.246369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff73ad5-66aa-48ba-8678-10c12259bd43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:01.246682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff73ad5-66aa-48ba-8678-10c12259bd43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:01.257425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff73ad5-66aa-48ba-8678-10c12259bd43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:01.258931 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff73ad5-66aa-48ba-8678-10c12259bd43 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:01.269940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1065/2125] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:01 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:01.348207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f2bb7-2171-4d1c-ae8b-bca5fd889fb6 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:01.358546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f2bb7-2171-4d1c-ae8b-bca5fd889fb6 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:01.362852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dae8f-9260-4cc1-9b26-f5113e57b94c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:01.363512 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dae8f-9260-4cc1-9b26-f5113e57b94c service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:01.363819 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dae8f-9260-4cc1-9b26-f5113e57b94c service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:01.364109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dae8f-9260-4cc1-9b26-f5113e57b94c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:01.391585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:01.431459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534dae8f-9260-4cc1-9b26-f5113e57b94c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:01.432966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534dae8f-9260-4cc1-9b26-f5113e57b94c service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:01.441794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1061/2126] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:01 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:11.265153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=51,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:11.437838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=17 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:19.866514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff73ad5-66aa-48ba-8678-10c12259bd43 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:19.927425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff73ad5-66aa-48ba-8678-10c12259bd43 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:19.931843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:19.932878 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:19.933160 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:19.933436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] PATH_INFO: `/v3/users/cebee58fba6e4efbafef8a179ed45cb8` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:19.934606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cebee58fba6e4efbafef8a179ed45cb8)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:19.936553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:19.980709 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cebee58fba6e4efbafef8a179ed45cb8'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:19.987562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cebee58fba6e4efbafef8a179ed45cb8'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:19.988210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cebee58fba6e4efbafef8a179ed45cb8'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:19.999998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cebee58fba6e4efbafef8a179ed45cb8'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.000678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cebee58fba6e4efbafef8a179ed45cb8'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.013252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1066/2127] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:19 2022] DELETE /identity/v3/users/cebee58fba6e4efbafef8a179ed45cb8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:20.017774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afa300-9d18-4ba2-a746-0e4877a17d58 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:20.018243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afa300-9d18-4ba2-a746-0e4877a17d58 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:20.018362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afa300-9d18-4ba2-a746-0e4877a17d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:20.035793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f7afa300-9d18-4ba2-a746-0e4877a17d58 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:20.037052 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f7afa300-9d18-4ba2-a746-0e4877a17d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:20.094339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1062/2128] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:20.103766 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:20.105231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:20.157800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afa300-9d18-4ba2-a746-0e4877a17d58 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:20.159479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f7afa300-9d18-4ba2-a746-0e4877a17d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:20.164111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cd629-3512-4998-96a1-b007348d2e12 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:20.169234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e19810-c05e-42ed-aa66-67cd1a939a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:20.169891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e19810-c05e-42ed-aa66-67cd1a939a25 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:20.170186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e19810-c05e-42ed-aa66-67cd1a939a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:20.170472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e19810-c05e-42ed-aa66-67cd1a939a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:20.174171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afa300-9d18-4ba2-a746-0e4877a17d58 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:20.178929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:20.179733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:20.180076 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:20.180398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] PATH_INFO: `/v3/users/0d095d34fd104238b9b75c9950d276af` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:20.181421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e19810-c05e-42ed-aa66-67cd1a939a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:20.181894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d095d34fd104238b9b75c9950d276af)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:20.182916 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e19810-c05e-42ed-aa66-67cd1a939a25 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:20.184365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:20.203875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1067/2129] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:20.248910 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d095d34fd104238b9b75c9950d276af'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.250474 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d095d34fd104238b9b75c9950d276af'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.258934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d095d34fd104238b9b75c9950d276af'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.274384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d095d34fd104238b9b75c9950d276af'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.275202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d095d34fd104238b9b75c9950d276af'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.292155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1063/2130] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:20 2022] DELETE /identity/v3/users/0d095d34fd104238b9b75c9950d276af => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:20.296421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b4f58-155d-42b8-ac3e-f05dddd41af0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:20.296848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b4f58-155d-42b8-ac3e-f05dddd41af0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:20.297281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b4f58-155d-42b8-ac3e-f05dddd41af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:20.334296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6f0b4f58-155d-42b8-ac3e-f05dddd41af0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:20.335683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6f0b4f58-155d-42b8-ac3e-f05dddd41af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:20.355769 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:20.424100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1068/2131] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:20.435765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:20.437319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:20.507590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537dec3-4d34-4855-8018-239a28046cff admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:20.515271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047858b4-b716-41d6-a02a-eaebcb064dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:20.516751 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047858b4-b716-41d6-a02a-eaebcb064dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:20.517056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047858b4-b716-41d6-a02a-eaebcb064dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:20.517454 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047858b4-b716-41d6-a02a-eaebcb064dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:20.528236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047858b4-b716-41d6-a02a-eaebcb064dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:20.529673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047858b4-b716-41d6-a02a-eaebcb064dd1 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:20.549924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1064/2132] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:20.562751 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b4f58-155d-42b8-ac3e-f05dddd41af0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:20.564513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6f0b4f58-155d-42b8-ac3e-f05dddd41af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:20.585834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b4f58-155d-42b8-ac3e-f05dddd41af0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:20.598683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:20.599566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:20.599847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:20.600148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] PATH_INFO: `/v3/projects/4eae22c650b74448ba9e8fc15e006a85` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:20.601497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eae22c650b74448ba9e8fc15e006a85)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:20.604644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:20.665239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eae22c650b74448ba9e8fc15e006a85', 'request_id': 'req-1f4439ff-7547-4c20-8955-b7ca948d2c05'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.672133 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eae22c650b74448ba9e8fc15e006a85', 'request_id': 'req-1f4439ff-7547-4c20-8955-b7ca948d2c05'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.682970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eae22c650b74448ba9e8fc15e006a85', 'request_id': 'req-1f4439ff-7547-4c20-8955-b7ca948d2c05'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.700969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] The token cache is being invalidate because project 4eae22c650b74448ba9e8fc15e006a85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:20.702679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.703487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.705561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1069/2133] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:20 2022] DELETE /identity/v3/projects/4eae22c650b74448ba9e8fc15e006a85 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:20.709395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047858b4-b716-41d6-a02a-eaebcb064dd1 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:20.719034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047858b4-b716-41d6-a02a-eaebcb064dd1 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:20.750987 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:20.810208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0198c4-f926-4360-b913-1439e9a64bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:20.811055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0198c4-f926-4360-b913-1439e9a64bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:20.811403 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0198c4-f926-4360-b913-1439e9a64bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:20.811741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0198c4-f926-4360-b913-1439e9a64bf7 admin admin] PATH_INFO: `/v3/users/1741fd4b714c43b793d023b010519d6b` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:20.813442 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0198c4-f926-4360-b913-1439e9a64bf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1741fd4b714c43b793d023b010519d6b)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:20.815433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0198c4-f926-4360-b913-1439e9a64bf7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:20.871479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:20.873935 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:20.894753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-2d0198c4-f926-4360-b913-1439e9a64bf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1741fd4b714c43b793d023b010519d6b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.895651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-2d0198c4-f926-4360-b913-1439e9a64bf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1741fd4b714c43b793d023b010519d6b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.904451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-2d0198c4-f926-4360-b913-1439e9a64bf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1741fd4b714c43b793d023b010519d6b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.907771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4439ff-7547-4c20-8955-b7ca948d2c05 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:20.915341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:20.916646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:20.917007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:20.917364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d admin admin] PATH_INFO: `/v3/projects/1a6acf47bea7483b81aeda9070e69537` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:20.918676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a6acf47bea7483b81aeda9070e69537)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:20.921054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:20.924962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-2d0198c4-f926-4360-b913-1439e9a64bf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1741fd4b714c43b793d023b010519d6b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.925941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-2d0198c4-f926-4360-b913-1439e9a64bf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1741fd4b714c43b793d023b010519d6b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.944026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1065/2134] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:20 2022] DELETE /identity/v3/users/1741fd4b714c43b793d023b010519d6b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:20.983050 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a6acf47bea7483b81aeda9070e69537', 'request_id': 'req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.989041 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a6acf47bea7483b81aeda9070e69537', 'request_id': 'req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:20.996733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a6acf47bea7483b81aeda9070e69537', 'request_id': 'req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:21.009183 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d admin admin] The token cache is being invalidate because project 1a6acf47bea7483b81aeda9070e69537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:21.009477 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:21.010180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-404a5f24-4a61-481d-87e4-7ea7ae83dd8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:21.012467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1070/2135] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:20 2022] DELETE /identity/v3/projects/1a6acf47bea7483b81aeda9070e69537 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:21.017831 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ef9bd-fd89-4bef-8b3e-2529e206f93c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.018345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ef9bd-fd89-4bef-8b3e-2529e206f93c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.018756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ef9bd-fd89-4bef-8b3e-2529e206f93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.038361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-845ef9bd-fd89-4bef-8b3e-2529e206f93c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:21.039562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-845ef9bd-fd89-4bef-8b3e-2529e206f93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:21.105301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1066/2136] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:21.113839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62530ab-4a81-43d3-910e-6c30a8d0ecff None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.114216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62530ab-4a81-43d3-910e-6c30a8d0ecff None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.114503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62530ab-4a81-43d3-910e-6c30a8d0ecff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.134890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-d62530ab-4a81-43d3-910e-6c30a8d0ecff None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:21.136307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d62530ab-4a81-43d3-910e-6c30a8d0ecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:21.144749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1071/2137] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:21.148774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ef9bd-fd89-4bef-8b3e-2529e206f93c None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.158799 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ef9bd-fd89-4bef-8b3e-2529e206f93c None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.163712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0851d2-7c07-4c17-a34d-2a4a7b5f2d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.164414 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0851d2-7c07-4c17-a34d-2a4a7b5f2d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.164703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0851d2-7c07-4c17-a34d-2a4a7b5f2d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.164983 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0851d2-7c07-4c17-a34d-2a4a7b5f2d4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.165720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0851d2-7c07-4c17-a34d-2a4a7b5f2d4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:21:21.166035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0851d2-7c07-4c17-a34d-2a4a7b5f2d4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.167767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0851d2-7c07-4c17-a34d-2a4a7b5f2d4c admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.180180 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1067/2138] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:21:21.184962 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62530ab-4a81-43d3-910e-6c30a8d0ecff None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.194450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62530ab-4a81-43d3-910e-6c30a8d0ecff None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.199227 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327675f9-21c7-4331-8cfb-6c983d317bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.199883 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327675f9-21c7-4331-8cfb-6c983d317bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.200193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327675f9-21c7-4331-8cfb-6c983d317bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.200473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327675f9-21c7-4331-8cfb-6c983d317bf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.201152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327675f9-21c7-4331-8cfb-6c983d317bf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.203053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327675f9-21c7-4331-8cfb-6c983d317bf7 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.223602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:21.233881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0851d2-7c07-4c17-a34d-2a4a7b5f2d4c admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.234130 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1072/2139] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:21 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:21.236035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327675f9-21c7-4331-8cfb-6c983d317bf7 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.236852 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-cb0851d2-7c07-4c17-a34d-2a4a7b5f2d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:21.251617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0851d2-7c07-4c17-a34d-2a4a7b5f2d4c admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.279080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:21.308817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327675f9-21c7-4331-8cfb-6c983d317bf7 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.314271 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac441954-6d67-49b2-8e47-d2463c0d9a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.314944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac441954-6d67-49b2-8e47-d2463c0d9a47 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.315231 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac441954-6d67-49b2-8e47-d2463c0d9a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.315509 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac441954-6d67-49b2-8e47-d2463c0d9a47 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.316296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac441954-6d67-49b2-8e47-d2463c0d9a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.318220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac441954-6d67-49b2-8e47-d2463c0d9a47 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.321314 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.322174 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.322459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.322792 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] PATH_INFO: `/v3/projects/fecce91f10f64b8989bdf657ed870a3e` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.323990 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fecce91f10f64b8989bdf657ed870a3e)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.326127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.339689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1073/2140] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:21 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:21.343677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac441954-6d67-49b2-8e47-d2463c0d9a47 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.356602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac441954-6d67-49b2-8e47-d2463c0d9a47 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.361623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665d7bb-95f2-4607-9cef-b1e1738b0a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.362420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665d7bb-95f2-4607-9cef-b1e1738b0a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.362713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665d7bb-95f2-4607-9cef-b1e1738b0a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.362993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665d7bb-95f2-4607-9cef-b1e1738b0a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.363729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665d7bb-95f2-4607-9cef-b1e1738b0a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.367608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665d7bb-95f2-4607-9cef-b1e1738b0a8b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.383090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1074/2141] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:21.383090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f3d84-3945-44a7-941c-724f9debee5c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.383090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f3d84-3945-44a7-941c-724f9debee5c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.383090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f3d84-3945-44a7-941c-724f9debee5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.391350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fecce91f10f64b8989bdf657ed870a3e', 'request_id': 'req-6969cd02-8ffd-4504-b997-ea9e4a88794d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:21.403261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fecce91f10f64b8989bdf657ed870a3e', 'request_id': 'req-6969cd02-8ffd-4504-b997-ea9e4a88794d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:21.406760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-604f3d84-3945-44a7-941c-724f9debee5c None None] MFA Rules not processed for user `9710f6fb31da4df39b04839a29dfdb34`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:21.408218 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-604f3d84-3945-44a7-941c-724f9debee5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:21.410487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fecce91f10f64b8989bdf657ed870a3e', 'request_id': 'req-6969cd02-8ffd-4504-b997-ea9e4a88794d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:21.421981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] The token cache is being invalidate because project fecce91f10f64b8989bdf657ed870a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:21.422340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:21.423061 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:21.427977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1068/2142] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:21 2022] DELETE /identity/v3/projects/fecce91f10f64b8989bdf657ed870a3e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:21.427977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.427977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:21.463268 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1075/2143] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:21.465541 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bc374-2ed0-4da3-9f8a-a8defe6f3072 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.465836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bc374-2ed0-4da3-9f8a-a8defe6f3072 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.466119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bc374-2ed0-4da3-9f8a-a8defe6f3072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.486401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-b12bc374-2ed0-4da3-9f8a-a8defe6f3072 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:21.487545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b12bc374-2ed0-4da3-9f8a-a8defe6f3072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:21.493942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969cd02-8ffd-4504-b997-ea9e4a88794d admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.497212 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1076/2144] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:21.504766 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55162776-eb7e-4ac9-9b73-fcccdb0fce36 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.505076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55162776-eb7e-4ac9-9b73-fcccdb0fce36 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.505218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37696e09-fa58-4c68-b00b-10fd2c6a7990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.505336 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55162776-eb7e-4ac9-9b73-fcccdb0fce36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.505776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37696e09-fa58-4c68-b00b-10fd2c6a7990 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.506059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37696e09-fa58-4c68-b00b-10fd2c6a7990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.506345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37696e09-fa58-4c68-b00b-10fd2c6a7990 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.507084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37696e09-fa58-4c68-b00b-10fd2c6a7990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.510699 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37696e09-fa58-4c68-b00b-10fd2c6a7990 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.522526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1069/2145] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:21.528802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-55162776-eb7e-4ac9-9b73-fcccdb0fce36 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:21.529025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37696e09-fa58-4c68-b00b-10fd2c6a7990 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.536634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-55162776-eb7e-4ac9-9b73-fcccdb0fce36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:21.539463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37696e09-fa58-4c68-b00b-10fd2c6a7990 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.549158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bf6e4-6258-4701-a9c0-738c8eaa0f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.549395 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1077/2146] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:21:21.550872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bf6e4-6258-4701-a9c0-738c8eaa0f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.551330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bf6e4-6258-4701-a9c0-738c8eaa0f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.551777 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bf6e4-6258-4701-a9c0-738c8eaa0f60 admin admin] PATH_INFO: `/v3/projects/564c2fe9f88748a2825a129fe97f7181/users/bcf5b641ef544274a7722607461fca20/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.553067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55162776-eb7e-4ac9-9b73-fcccdb0fce36 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.563588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55162776-eb7e-4ac9-9b73-fcccdb0fce36 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.570155 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3991f-9f53-4e9e-b67c-779753d16bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.570857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3991f-9f53-4e9e-b67c-779753d16bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.571145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3991f-9f53-4e9e-b67c-779753d16bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.571421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3991f-9f53-4e9e-b67c-779753d16bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.572442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3991f-9f53-4e9e-b67c-779753d16bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:21:21.573195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3991f-9f53-4e9e-b67c-779753d16bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.574656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bf6e4-6258-4701-a9c0-738c8eaa0f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=564c2fe9f88748a2825a129fe97f7181, user_id=bcf5b641ef544274a7722607461fca20, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.574786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3991f-9f53-4e9e-b67c-779753d16bc5 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.577919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bf6e4-6258-4701-a9c0-738c8eaa0f60 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.595299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1078/2147] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:21:21.598708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1070/2148] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:21 2022] PUT /identity/v3/projects/564c2fe9f88748a2825a129fe97f7181/users/bcf5b641ef544274a7722607461fca20/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:21.607594 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3991f-9f53-4e9e-b67c-779753d16bc5 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.608415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4f22f-56b7-4c3a-95c3-6aca1eb28b5d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.608792 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4f22f-56b7-4c3a-95c3-6aca1eb28b5d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.609075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4f22f-56b7-4c3a-95c3-6aca1eb28b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.618789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3991f-9f53-4e9e-b67c-779753d16bc5 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.651781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-23e4f22f-56b7-4c3a-95c3-6aca1eb28b5d None None] MFA Rules not processed for user `bcf5b641ef544274a7722607461fca20`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:21.654212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-23e4f22f-56b7-4c3a-95c3-6aca1eb28b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:21.670262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:21.683211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:21.690209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adbfc9-03d6-4374-9067-675f0e203437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.691056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adbfc9-03d6-4374-9067-675f0e203437 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.691175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adbfc9-03d6-4374-9067-675f0e203437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.691447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adbfc9-03d6-4374-9067-675f0e203437 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.692422 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3adbfc9-03d6-4374-9067-675f0e203437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.702355 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3adbfc9-03d6-4374-9067-675f0e203437 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.732701 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1079/2149] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:21 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:21.736801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adbfc9-03d6-4374-9067-675f0e203437 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.747782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1071/2150] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:21:21.755558 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4f22f-56b7-4c3a-95c3-6aca1eb28b5d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.756820 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-23e4f22f-56b7-4c3a-95c3-6aca1eb28b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:21.809137 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adbfc9-03d6-4374-9067-675f0e203437 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.814162 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ecd5a-d7d8-40dc-adb7-05f7407859b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.814832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ecd5a-d7d8-40dc-adb7-05f7407859b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.815115 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ecd5a-d7d8-40dc-adb7-05f7407859b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.815392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ecd5a-d7d8-40dc-adb7-05f7407859b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.816101 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ecd5a-d7d8-40dc-adb7-05f7407859b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.818043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ecd5a-d7d8-40dc-adb7-05f7407859b0 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.826297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4f22f-56b7-4c3a-95c3-6aca1eb28b5d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.841967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1080/2151] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:21 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:21:21.843091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ecd5a-d7d8-40dc-adb7-05f7407859b0 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.859356 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ecd5a-d7d8-40dc-adb7-05f7407859b0 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.864251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285af9c-ed4c-4a2f-b7e2-9cddc0ce2b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.864951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285af9c-ed4c-4a2f-b7e2-9cddc0ce2b56 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.865367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285af9c-ed4c-4a2f-b7e2-9cddc0ce2b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.865655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285af9c-ed4c-4a2f-b7e2-9cddc0ce2b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.866392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285af9c-ed4c-4a2f-b7e2-9cddc0ce2b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.868014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285af9c-ed4c-4a2f-b7e2-9cddc0ce2b56 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.880714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1081/2152] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:21:21.883718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285af9c-ed4c-4a2f-b7e2-9cddc0ce2b56 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.892560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ed9fd-0aab-4ea9-be70-7464eb868375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.893216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ed9fd-0aab-4ea9-be70-7464eb868375 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.893499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ed9fd-0aab-4ea9-be70-7464eb868375 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.893793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ed9fd-0aab-4ea9-be70-7464eb868375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.894120 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285af9c-ed4c-4a2f-b7e2-9cddc0ce2b56 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.899003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca44b15-9222-4eb4-b1db-ac8598860210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.899669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca44b15-9222-4eb4-b1db-ac8598860210 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.899958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca44b15-9222-4eb4-b1db-ac8598860210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.900378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca44b15-9222-4eb4-b1db-ac8598860210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.901138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca44b15-9222-4eb4-b1db-ac8598860210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.902761 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca44b15-9222-4eb4-b1db-ac8598860210 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.916121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1082/2153] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:21:21.919317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca44b15-9222-4eb4-b1db-ac8598860210 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:21.929788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca44b15-9222-4eb4-b1db-ac8598860210 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:21.939580 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17615b6c-8313-4c24-a32c-d5d345f9d505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:21.941144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17615b6c-8313-4c24-a32c-d5d345f9d505 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.941436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17615b6c-8313-4c24-a32c-d5d345f9d505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.941717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17615b6c-8313-4c24-a32c-d5d345f9d505 admin admin] PATH_INFO: `/v3/projects/f80fc227328046ed9a20c8d419d3ed10/users/cefbfdba891645819cb112d5de948ab4/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.960679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17615b6c-8313-4c24-a32c-d5d345f9d505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f80fc227328046ed9a20c8d419d3ed10, user_id=cefbfdba891645819cb112d5de948ab4, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.963731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17615b6c-8313-4c24-a32c-d5d345f9d505 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.974488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ed9fd-0aab-4ea9-be70-7464eb868375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:21.975936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ed9fd-0aab-4ea9-be70-7464eb868375 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:21.980568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1083/2154] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:21 2022] PUT /identity/v3/projects/f80fc227328046ed9a20c8d419d3ed10/users/cefbfdba891645819cb112d5de948ab4/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:21.991720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c538a-dcf8-40b9-b5ee-cc3dccb1b74c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:21.991943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c538a-dcf8-40b9-b5ee-cc3dccb1b74c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:21.992257 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c538a-dcf8-40b9-b5ee-cc3dccb1b74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:21.997975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1072/2155] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:21:22.027265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-743c538a-dcf8-40b9-b5ee-cc3dccb1b74c None None] MFA Rules not processed for user `cefbfdba891645819cb112d5de948ab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:22.028606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-743c538a-dcf8-40b9-b5ee-cc3dccb1b74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:22.124879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1084/2156] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:21 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:22.129238 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ed9fd-0aab-4ea9-be70-7464eb868375 service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:22.149485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ed9fd-0aab-4ea9-be70-7464eb868375 service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:22.171521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:22.223471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ee757-0c1d-4daf-8554-2bfd68148a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:22.228600 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ee757-0c1d-4daf-8554-2bfd68148a22 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:22.228992 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ee757-0c1d-4daf-8554-2bfd68148a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:22.229354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ee757-0c1d-4daf-8554-2bfd68148a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:22.230080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ee757-0c1d-4daf-8554-2bfd68148a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:22.232268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ee757-0c1d-4daf-8554-2bfd68148a22 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:22.261874 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1073/2157] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:22 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:22.266322 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c538a-dcf8-40b9-b5ee-cc3dccb1b74c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:22.281525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c538a-dcf8-40b9-b5ee-cc3dccb1b74c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:22.296389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:22.371345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba248988-ec22-4f4b-8a81-4c37a8e32cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:22.374971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba248988-ec22-4f4b-8a81-4c37a8e32cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:22.375283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba248988-ec22-4f4b-8a81-4c37a8e32cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:22.375658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba248988-ec22-4f4b-8a81-4c37a8e32cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:22.376416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba248988-ec22-4f4b-8a81-4c37a8e32cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:22.378551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba248988-ec22-4f4b-8a81-4c37a8e32cd0 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:22.406573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1085/2158] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:22 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:22.410473 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ee757-0c1d-4daf-8554-2bfd68148a22 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:22.428667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ee757-0c1d-4daf-8554-2bfd68148a22 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:22.433140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070f3f-3bd5-4f70-ba92-419e450dcf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:22.433834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070f3f-3bd5-4f70-ba92-419e450dcf54 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:22.434118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070f3f-3bd5-4f70-ba92-419e450dcf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:22.434394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070f3f-3bd5-4f70-ba92-419e450dcf54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:22.435129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40070f3f-3bd5-4f70-ba92-419e450dcf54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:22.437015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40070f3f-3bd5-4f70-ba92-419e450dcf54 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:22.452117 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1074/2159] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:22.455675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba248988-ec22-4f4b-8a81-4c37a8e32cd0 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:22.468111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba248988-ec22-4f4b-8a81-4c37a8e32cd0 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:22.474243 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88fb9e-c6a1-41c5-9767-9f9582fec633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:22.475785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88fb9e-c6a1-41c5-9767-9f9582fec633 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:22.476222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88fb9e-c6a1-41c5-9767-9f9582fec633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:22.476551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88fb9e-c6a1-41c5-9767-9f9582fec633 admin admin] PATH_INFO: `/v3/projects/d7916deb4cd04af99c3d747a15693500/users/115c5927662f46a49a2de7e10ff391bc/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:22.494276 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88fb9e-c6a1-41c5-9767-9f9582fec633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7916deb4cd04af99c3d747a15693500, user_id=115c5927662f46a49a2de7e10ff391bc, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:22.497916 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88fb9e-c6a1-41c5-9767-9f9582fec633 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:22.510952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1086/2160] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:22 2022] PUT /identity/v3/projects/d7916deb4cd04af99c3d747a15693500/users/115c5927662f46a49a2de7e10ff391bc/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:22.514864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070f3f-3bd5-4f70-ba92-419e450dcf54 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:22.524540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070f3f-3bd5-4f70-ba92-419e450dcf54 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:22.580344 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b1ec5-04e3-428d-aa04-d0d83e7901a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:22.581004 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b1ec5-04e3-428d-aa04-d0d83e7901a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:22.581289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b1ec5-04e3-428d-aa04-d0d83e7901a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:22.581698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b1ec5-04e3-428d-aa04-d0d83e7901a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:22.582280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b1ec5-04e3-428d-aa04-d0d83e7901a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:22.583873 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b1ec5-04e3-428d-aa04-d0d83e7901a7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:22.593946 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1075/2161] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:22.598114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88fb9e-c6a1-41c5-9767-9f9582fec633 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:22.608830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88fb9e-c6a1-41c5-9767-9f9582fec633 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:22.613617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471539f-f828-4e3c-995e-0c5719a1021e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:22.614548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471539f-f828-4e3c-995e-0c5719a1021e admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:22.614835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471539f-f828-4e3c-995e-0c5719a1021e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:22.615116 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471539f-f828-4e3c-995e-0c5719a1021e admin admin] PATH_INFO: `/v3/projects/d7916deb4cd04af99c3d747a15693500/users/115c5927662f46a49a2de7e10ff391bc/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:22.617150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471539f-f828-4e3c-995e-0c5719a1021e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7916deb4cd04af99c3d747a15693500, user_id=115c5927662f46a49a2de7e10ff391bc, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:22.619835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471539f-f828-4e3c-995e-0c5719a1021e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:22.629902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1087/2162] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:22 2022] PUT /identity/v3/projects/d7916deb4cd04af99c3d747a15693500/users/115c5927662f46a49a2de7e10ff391bc/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:22.639858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe257fa2-3049-4da0-af78-4cec6d38ba17 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:22.640202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe257fa2-3049-4da0-af78-4cec6d38ba17 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:22.640490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe257fa2-3049-4da0-af78-4cec6d38ba17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:22.667638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-fe257fa2-3049-4da0-af78-4cec6d38ba17 None None] MFA Rules not processed for user `115c5927662f46a49a2de7e10ff391bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:22.668858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-fe257fa2-3049-4da0-af78-4cec6d38ba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:22.739977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1076/2163] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:22 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:22.751653 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471539f-f828-4e3c-995e-0c5719a1021e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:22.753559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f471539f-f828-4e3c-995e-0c5719a1021e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:22.811064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471539f-f828-4e3c-995e-0c5719a1021e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:22.814754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d41ee-060d-4feb-934d-991085c3cdae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:22.815416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d41ee-060d-4feb-934d-991085c3cdae service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:22.815702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d41ee-060d-4feb-934d-991085c3cdae service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:22.816033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d41ee-060d-4feb-934d-991085c3cdae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:22.825311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d41ee-060d-4feb-934d-991085c3cdae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:22.826764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d41ee-060d-4feb-934d-991085c3cdae service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:22.836624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1088/2164] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:22 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:22.880540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe257fa2-3049-4da0-af78-4cec6d38ba17 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:22.890614 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe257fa2-3049-4da0-af78-4cec6d38ba17 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:22.894172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1fab3-7c41-4faf-8fb1-549e090f1003 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:22.894803 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1fab3-7c41-4faf-8fb1-549e090f1003 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:22.895086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1fab3-7c41-4faf-8fb1-549e090f1003 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:22.895363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1fab3-7c41-4faf-8fb1-549e090f1003 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:22.913091 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:22.952666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1fab3-7c41-4faf-8fb1-549e090f1003 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:22.954084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1fab3-7c41-4faf-8fb1-549e090f1003 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:22.962746 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1077/2165] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:26.482741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d41ee-060d-4feb-934d-991085c3cdae service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:26.549526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d41ee-060d-4feb-934d-991085c3cdae service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:26.554113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:26.555117 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:26.555411 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:26.555718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] PATH_INFO: `/v3/users/cefbfdba891645819cb112d5de948ab4` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:26.556955 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cefbfdba891645819cb112d5de948ab4)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:26.558866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:26.571250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:26.606714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cefbfdba891645819cb112d5de948ab4'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:26.613703 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cefbfdba891645819cb112d5de948ab4'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:26.614425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cefbfdba891645819cb112d5de948ab4'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:26.626035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cefbfdba891645819cb112d5de948ab4'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:26.626663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cefbfdba891645819cb112d5de948ab4'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:26.639446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1089/2166] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:26 2022] DELETE /identity/v3/users/cefbfdba891645819cb112d5de948ab4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:26.643940 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acbb7f-b005-40f4-8fdd-190f781cafd6 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:26.644401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acbb7f-b005-40f4-8fdd-190f781cafd6 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:26.644588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acbb7f-b005-40f4-8fdd-190f781cafd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:26.662787 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-78acbb7f-b005-40f4-8fdd-190f781cafd6 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:26.663986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-78acbb7f-b005-40f4-8fdd-190f781cafd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:26.725109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1078/2167] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:21:26.734364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:26.735741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:26.798583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6825e-f177-4f6e-b1e6-0acfd1d96afd admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:26.802378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df8ab7-f696-4fe4-b41b-40a37ddd3588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:26.802973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df8ab7-f696-4fe4-b41b-40a37ddd3588 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:26.803262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df8ab7-f696-4fe4-b41b-40a37ddd3588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:26.803596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df8ab7-f696-4fe4-b41b-40a37ddd3588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:26.813911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df8ab7-f696-4fe4-b41b-40a37ddd3588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:26.815348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df8ab7-f696-4fe4-b41b-40a37ddd3588 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:26.824932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1090/2168] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:27.089477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78acbb7f-b005-40f4-8fdd-190f781cafd6 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:27.090864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-78acbb7f-b005-40f4-8fdd-190f781cafd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:27.105488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78acbb7f-b005-40f4-8fdd-190f781cafd6 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:27.111044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:27.112155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:27.112486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:27.112772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] PATH_INFO: `/v3/projects/f80fc227328046ed9a20c8d419d3ed10` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:27.113916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f80fc227328046ed9a20c8d419d3ed10)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:27.115873 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:27.163218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f80fc227328046ed9a20c8d419d3ed10', 'request_id': 'req-69f3ae2a-53b7-443c-92fc-05194766ac92'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.171581 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f80fc227328046ed9a20c8d419d3ed10', 'request_id': 'req-69f3ae2a-53b7-443c-92fc-05194766ac92'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.177737 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f80fc227328046ed9a20c8d419d3ed10', 'request_id': 'req-69f3ae2a-53b7-443c-92fc-05194766ac92'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.187744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] The token cache is being invalidate because project f80fc227328046ed9a20c8d419d3ed10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:27.188037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.188680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.190200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1079/2169] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:27 2022] DELETE /identity/v3/projects/f80fc227328046ed9a20c8d419d3ed10 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:27.194185 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df8ab7-f696-4fe4-b41b-40a37ddd3588 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:27.203823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df8ab7-f696-4fe4-b41b-40a37ddd3588 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:27.258680 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e89a0-7e37-42ed-a95c-d19255a404f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:27.259549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e89a0-7e37-42ed-a95c-d19255a404f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:27.259834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e89a0-7e37-42ed-a95c-d19255a404f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:27.260732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e89a0-7e37-42ed-a95c-d19255a404f5 admin admin] PATH_INFO: `/v3/users/115c5927662f46a49a2de7e10ff391bc` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:27.261936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e89a0-7e37-42ed-a95c-d19255a404f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=115c5927662f46a49a2de7e10ff391bc)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:27.263866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e89a0-7e37-42ed-a95c-d19255a404f5 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:27.298583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:27.309698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b29e89a0-7e37-42ed-a95c-d19255a404f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '115c5927662f46a49a2de7e10ff391bc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.316416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b29e89a0-7e37-42ed-a95c-d19255a404f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '115c5927662f46a49a2de7e10ff391bc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.317043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b29e89a0-7e37-42ed-a95c-d19255a404f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '115c5927662f46a49a2de7e10ff391bc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.329155 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b29e89a0-7e37-42ed-a95c-d19255a404f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '115c5927662f46a49a2de7e10ff391bc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.329777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b29e89a0-7e37-42ed-a95c-d19255a404f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '115c5927662f46a49a2de7e10ff391bc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.341736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1091/2170] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:27 2022] DELETE /identity/v3/users/115c5927662f46a49a2de7e10ff391bc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:27.691660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:27.692943 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:27.708025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3ae2a-53b7-443c-92fc-05194766ac92 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:27.759570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:27.787026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:27.787026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:27.787026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:27.787026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2 admin admin] PATH_INFO: `/v3/projects/d7916deb4cd04af99c3d747a15693500` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:27.787838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7916deb4cd04af99c3d747a15693500)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:27.787838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:27.842045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7916deb4cd04af99c3d747a15693500', 'request_id': 'req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.852667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7916deb4cd04af99c3d747a15693500', 'request_id': 'req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.862124 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7916deb4cd04af99c3d747a15693500', 'request_id': 'req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.877270 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2 admin admin] The token cache is being invalidate because project d7916deb4cd04af99c3d747a15693500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:27.877566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.878856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f2eef8f9-8f08-4e7e-a4e5-bf5d2a623ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:27.881719 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1080/2171] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:27 2022] DELETE /identity/v3/projects/d7916deb4cd04af99c3d747a15693500 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:27.886990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96607999-7ea3-4cc1-9289-df104e363106 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:27.887481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96607999-7ea3-4cc1-9289-df104e363106 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:27.887562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96607999-7ea3-4cc1-9289-df104e363106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:27.908517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-96607999-7ea3-4cc1-9289-df104e363106 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:27.909922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-96607999-7ea3-4cc1-9289-df104e363106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:28.000875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1092/2172] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:28.009017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e808dd-9f5a-45cb-98f4-482583b45f68 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.009504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e808dd-9f5a-45cb-98f4-482583b45f68 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.009812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e808dd-9f5a-45cb-98f4-482583b45f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.043731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-12e808dd-9f5a-45cb-98f4-482583b45f68 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:28.045130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-12e808dd-9f5a-45cb-98f4-482583b45f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:28.061510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1081/2173] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:28.065579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96607999-7ea3-4cc1-9289-df104e363106 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:28.077804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96607999-7ea3-4cc1-9289-df104e363106 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:28.082610 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd67a5e-f65f-45e9-91bc-32823ed72130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:28.083392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd67a5e-f65f-45e9-91bc-32823ed72130 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.083685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd67a5e-f65f-45e9-91bc-32823ed72130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.083969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd67a5e-f65f-45e9-91bc-32823ed72130 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.085730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd67a5e-f65f-45e9-91bc-32823ed72130 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:21:28.086048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd67a5e-f65f-45e9-91bc-32823ed72130 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:28.087664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd67a5e-f65f-45e9-91bc-32823ed72130 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:28.103410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1093/2174] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:21:28.105015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e808dd-9f5a-45cb-98f4-482583b45f68 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:28.114934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e808dd-9f5a-45cb-98f4-482583b45f68 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:28.119724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27840361-41d2-4a5e-a694-249a09f13e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:28.120461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27840361-41d2-4a5e-a694-249a09f13e8c admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.120743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27840361-41d2-4a5e-a694-249a09f13e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.121020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27840361-41d2-4a5e-a694-249a09f13e8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.121712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27840361-41d2-4a5e-a694-249a09f13e8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:28.123608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27840361-41d2-4a5e-a694-249a09f13e8c admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:28.160102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1082/2175] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:28 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:21:28.161293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd67a5e-f65f-45e9-91bc-32823ed72130 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:28.172713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd67a5e-f65f-45e9-91bc-32823ed72130 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:28.234788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d71db-3bbb-4faa-8377-7ddfebefa542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:28.235357 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d71db-3bbb-4faa-8377-7ddfebefa542 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.235646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d71db-3bbb-4faa-8377-7ddfebefa542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.235927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d71db-3bbb-4faa-8377-7ddfebefa542 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.236648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d71db-3bbb-4faa-8377-7ddfebefa542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:28.238606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d71db-3bbb-4faa-8377-7ddfebefa542 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:28.263625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1094/2176] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:28 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:28.271123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27840361-41d2-4a5e-a694-249a09f13e8c admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:28.289745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27840361-41d2-4a5e-a694-249a09f13e8c admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:28.295281 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e75a5-1b74-4c99-a6c7-12880efefc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:28.295948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e75a5-1b74-4c99-a6c7-12880efefc33 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.296260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e75a5-1b74-4c99-a6c7-12880efefc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.296536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e75a5-1b74-4c99-a6c7-12880efefc33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.297266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e75a5-1b74-4c99-a6c7-12880efefc33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:28.298874 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998e75a5-1b74-4c99-a6c7-12880efefc33 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:28.310341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1083/2177] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:28.314296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d71db-3bbb-4faa-8377-7ddfebefa542 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:28.325987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d71db-3bbb-4faa-8377-7ddfebefa542 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:28.330952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de5b82-cfb7-4157-9869-2800ee4de8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:28.331617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de5b82-cfb7-4157-9869-2800ee4de8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.331910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de5b82-cfb7-4157-9869-2800ee4de8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.332233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de5b82-cfb7-4157-9869-2800ee4de8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.333141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de5b82-cfb7-4157-9869-2800ee4de8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:28.334759 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de5b82-cfb7-4157-9869-2800ee4de8d1 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:28.345082 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1095/2178] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:28.349183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e75a5-1b74-4c99-a6c7-12880efefc33 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:28.359693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e75a5-1b74-4c99-a6c7-12880efefc33 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:28.365974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e61ac-64cd-4aca-81ec-910806fb4094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:28.367026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e61ac-64cd-4aca-81ec-910806fb4094 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.367447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e61ac-64cd-4aca-81ec-910806fb4094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.367883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e61ac-64cd-4aca-81ec-910806fb4094 admin admin] PATH_INFO: `/v3/projects/fe1f21ceb43f4fce9bf102ed7fbfb709/users/3d39e43cb0964fdd920a10ccb6249d29/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.384995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e61ac-64cd-4aca-81ec-910806fb4094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1f21ceb43f4fce9bf102ed7fbfb709, user_id=3d39e43cb0964fdd920a10ccb6249d29, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:28.388359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e61ac-64cd-4aca-81ec-910806fb4094 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:28.407204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1084/2179] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:28 2022] PUT /identity/v3/projects/fe1f21ceb43f4fce9bf102ed7fbfb709/users/3d39e43cb0964fdd920a10ccb6249d29/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:28.419802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dffb9-ff57-471e-a289-00e7b8eb7974 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.420216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dffb9-ff57-471e-a289-00e7b8eb7974 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.420571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dffb9-ff57-471e-a289-00e7b8eb7974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.447431 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-923dffb9-ff57-471e-a289-00e7b8eb7974 None None] MFA Rules not processed for user `3d39e43cb0964fdd920a10ccb6249d29`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:28.448789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-923dffb9-ff57-471e-a289-00e7b8eb7974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:28.453750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:28.515585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1096/2180] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:28.519923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e61ac-64cd-4aca-81ec-910806fb4094 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:28.587980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e61ac-64cd-4aca-81ec-910806fb4094 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:28.594641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54495d84-c23b-48c4-a0fa-b0da716542cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:28.595311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54495d84-c23b-48c4-a0fa-b0da716542cb admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.595604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54495d84-c23b-48c4-a0fa-b0da716542cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.595885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54495d84-c23b-48c4-a0fa-b0da716542cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.596634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54495d84-c23b-48c4-a0fa-b0da716542cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:28.598558 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54495d84-c23b-48c4-a0fa-b0da716542cb admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:28.615914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:28.630330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1085/2181] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:28 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:28.634225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dffb9-ff57-471e-a289-00e7b8eb7974 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:28.645111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dffb9-ff57-471e-a289-00e7b8eb7974 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:28.704160 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d2fa0-f881-4e0e-8501-a14668de29bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:28.704908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d2fa0-f881-4e0e-8501-a14668de29bb admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.705260 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d2fa0-f881-4e0e-8501-a14668de29bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.705602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d2fa0-f881-4e0e-8501-a14668de29bb admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.706357 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d2fa0-f881-4e0e-8501-a14668de29bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:28.708364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d2fa0-f881-4e0e-8501-a14668de29bb admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:28.733861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1097/2182] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:28 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:28.735130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54495d84-c23b-48c4-a0fa-b0da716542cb admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:28.746470 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54495d84-c23b-48c4-a0fa-b0da716542cb admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:28.758121 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278ec25-85c2-4762-99a3-729b9446cd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:28.758121 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278ec25-85c2-4762-99a3-729b9446cd0b admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.758121 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278ec25-85c2-4762-99a3-729b9446cd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.758121 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278ec25-85c2-4762-99a3-729b9446cd0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.758496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278ec25-85c2-4762-99a3-729b9446cd0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:28.758496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278ec25-85c2-4762-99a3-729b9446cd0b admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:28.770951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1086/2183] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:28.775286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d2fa0-f881-4e0e-8501-a14668de29bb admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:28.793992 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d2fa0-f881-4e0e-8501-a14668de29bb admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:28.799764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81f2a5-a97d-4cc9-9897-cd231b929fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:28.801100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81f2a5-a97d-4cc9-9897-cd231b929fad admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.801565 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81f2a5-a97d-4cc9-9897-cd231b929fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.802010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81f2a5-a97d-4cc9-9897-cd231b929fad admin admin] PATH_INFO: `/v3/projects/43bc5e3de097424789b4da0d68d040a1/users/542da5064827434c93cf287a33f80ff9/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.822204 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81f2a5-a97d-4cc9-9897-cd231b929fad admin admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5e3de097424789b4da0d68d040a1, user_id=542da5064827434c93cf287a33f80ff9, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:28.825024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81f2a5-a97d-4cc9-9897-cd231b929fad admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:28.849605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1098/2184] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:28 2022] PUT /identity/v3/projects/43bc5e3de097424789b4da0d68d040a1/users/542da5064827434c93cf287a33f80ff9/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:28.853530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278ec25-85c2-4762-99a3-729b9446cd0b admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:28.870619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278ec25-85c2-4762-99a3-729b9446cd0b admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:28.938498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20457d7f-1b99-4f9e-be2f-28b4a4b25d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:28.939703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20457d7f-1b99-4f9e-be2f-28b4a4b25d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.940106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20457d7f-1b99-4f9e-be2f-28b4a4b25d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.940486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20457d7f-1b99-4f9e-be2f-28b4a4b25d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.941371 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20457d7f-1b99-4f9e-be2f-28b4a4b25d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:28.943139 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20457d7f-1b99-4f9e-be2f-28b4a4b25d61 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:28.959618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1087/2185] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:28.960998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81f2a5-a97d-4cc9-9897-cd231b929fad admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:28.974199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81f2a5-a97d-4cc9-9897-cd231b929fad admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:28.983296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35295aa-23bb-4a08-8914-43e846442735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:28.984362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35295aa-23bb-4a08-8914-43e846442735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:28.984671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35295aa-23bb-4a08-8914-43e846442735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:28.984950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35295aa-23bb-4a08-8914-43e846442735 admin admin] PATH_INFO: `/v3/projects/43bc5e3de097424789b4da0d68d040a1/users/542da5064827434c93cf287a33f80ff9/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:28.987024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35295aa-23bb-4a08-8914-43e846442735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5e3de097424789b4da0d68d040a1, user_id=542da5064827434c93cf287a33f80ff9, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:28.990239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35295aa-23bb-4a08-8914-43e846442735 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:29.001773 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1099/2186] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:28 2022] PUT /identity/v3/projects/43bc5e3de097424789b4da0d68d040a1/users/542da5064827434c93cf287a33f80ff9/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:29.013302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcef5e3-23c8-4258-86cc-ffd9ee753a1d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:29.013634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcef5e3-23c8-4258-86cc-ffd9ee753a1d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:29.013909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcef5e3-23c8-4258-86cc-ffd9ee753a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:29.054253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-dbcef5e3-23c8-4258-86cc-ffd9ee753a1d None None] MFA Rules not processed for user `542da5064827434c93cf287a33f80ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:29.055699 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-dbcef5e3-23c8-4258-86cc-ffd9ee753a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:29.147529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1088/2187] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:29.159497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35295aa-23bb-4a08-8914-43e846442735 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:29.161345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c35295aa-23bb-4a08-8914-43e846442735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:29.223524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35295aa-23bb-4a08-8914-43e846442735 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:29.227306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21dff9b-9c43-4d27-b2a8-827d2f3881a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:29.228026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dff9b-9c43-4d27-b2a8-827d2f3881a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:29.228417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dff9b-9c43-4d27-b2a8-827d2f3881a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:29.228789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dff9b-9c43-4d27-b2a8-827d2f3881a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:29.237763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21dff9b-9c43-4d27-b2a8-827d2f3881a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:29.239267 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21dff9b-9c43-4d27-b2a8-827d2f3881a8 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:29.251004 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1100/2188] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:29.298672 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcef5e3-23c8-4258-86cc-ffd9ee753a1d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:29.308941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcef5e3-23c8-4258-86cc-ffd9ee753a1d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:29.312572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f783d8-7ece-4218-bf10-3e992f3e762e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:29.313198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f783d8-7ece-4218-bf10-3e992f3e762e service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:29.313481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f783d8-7ece-4218-bf10-3e992f3e762e service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:29.313755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f783d8-7ece-4218-bf10-3e992f3e762e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:29.373104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f783d8-7ece-4218-bf10-3e992f3e762e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:29.374475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f783d8-7ece-4218-bf10-3e992f3e762e service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:29.383804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1089/2189] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:29.870446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21dff9b-9c43-4d27-b2a8-827d2f3881a8 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:29.872044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d21dff9b-9c43-4d27-b2a8-827d2f3881a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:29.877852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:29.940043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21dff9b-9c43-4d27-b2a8-827d2f3881a8 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:29.944571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:29.945437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:29.945716 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:29.946002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] PATH_INFO: `/v3/users/d22fbcce79f94e3bbfe39730b609985f` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:29.947160 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d22fbcce79f94e3bbfe39730b609985f)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:29.949713 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:30.003209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd22fbcce79f94e3bbfe39730b609985f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:30.011711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd22fbcce79f94e3bbfe39730b609985f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:30.012386 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd22fbcce79f94e3bbfe39730b609985f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:30.025119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd22fbcce79f94e3bbfe39730b609985f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:30.025798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd22fbcce79f94e3bbfe39730b609985f'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:30.038301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1101/2190] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:29 2022] DELETE /identity/v3/users/d22fbcce79f94e3bbfe39730b609985f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:30.042795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275cc1e-f994-434f-9ce4-4dd50245143d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:30.043259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275cc1e-f994-434f-9ce4-4dd50245143d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:30.043351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275cc1e-f994-434f-9ce4-4dd50245143d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:30.049236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:30.064383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-2275cc1e-f994-434f-9ce4-4dd50245143d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:30.065972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-2275cc1e-f994-434f-9ce4-4dd50245143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:30.129438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1090/2191] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:21:30.140376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:30.142379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:30.203850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bd1e2-2e51-499e-93df-66a50a52179b admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:30.207462 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b72d7-bb72-461d-b426-7029a3ae745d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:30.208130 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b72d7-bb72-461d-b426-7029a3ae745d service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:30.208508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b72d7-bb72-461d-b426-7029a3ae745d service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:30.208678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b72d7-bb72-461d-b426-7029a3ae745d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:30.218283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b72d7-bb72-461d-b426-7029a3ae745d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:30.219885 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b72d7-bb72-461d-b426-7029a3ae745d service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:30.230059 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1102/2192] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:30.589495 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275cc1e-f994-434f-9ce4-4dd50245143d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:30.591053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-2275cc1e-f994-434f-9ce4-4dd50245143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:30.606445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275cc1e-f994-434f-9ce4-4dd50245143d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:30.611115 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13035b0f-fafa-4601-ab91-c87a48cbbe8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:30.611990 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13035b0f-fafa-4601-ab91-c87a48cbbe8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:30.612309 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13035b0f-fafa-4601-ab91-c87a48cbbe8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:30.612607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13035b0f-fafa-4601-ab91-c87a48cbbe8c admin admin] PATH_INFO: `/v3/projects/bf062d23afb54047835a2f166a938f02` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:30.614118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13035b0f-fafa-4601-ab91-c87a48cbbe8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf062d23afb54047835a2f166a938f02)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:30.615789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13035b0f-fafa-4601-ab91-c87a48cbbe8c admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:30.667599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-13035b0f-fafa-4601-ab91-c87a48cbbe8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf062d23afb54047835a2f166a938f02', 'request_id': 'req-13035b0f-fafa-4601-ab91-c87a48cbbe8c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:30.674801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-13035b0f-fafa-4601-ab91-c87a48cbbe8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf062d23afb54047835a2f166a938f02', 'request_id': 'req-13035b0f-fafa-4601-ab91-c87a48cbbe8c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:30.681024 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-13035b0f-fafa-4601-ab91-c87a48cbbe8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf062d23afb54047835a2f166a938f02', 'request_id': 'req-13035b0f-fafa-4601-ab91-c87a48cbbe8c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:30.691125 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-13035b0f-fafa-4601-ab91-c87a48cbbe8c admin admin] The token cache is being invalidate because project bf062d23afb54047835a2f166a938f02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:30.692740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-13035b0f-fafa-4601-ab91-c87a48cbbe8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:30.692740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-13035b0f-fafa-4601-ab91-c87a48cbbe8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:30.693645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1091/2193] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:30 2022] DELETE /identity/v3/projects/bf062d23afb54047835a2f166a938f02 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:30.698816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e264a-b125-4250-9c73-4dbba961d12f None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:30.699249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e264a-b125-4250-9c73-4dbba961d12f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:30.699704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e264a-b125-4250-9c73-4dbba961d12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:30.717450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-697e264a-b125-4250-9c73-4dbba961d12f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:30.718711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-697e264a-b125-4250-9c73-4dbba961d12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:30.743093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:30.784026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1103/2194] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:30.793539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7d39d-0299-45e7-b2c3-43dec546472a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:30.794102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7d39d-0299-45e7-b2c3-43dec546472a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:30.794246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7d39d-0299-45e7-b2c3-43dec546472a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:30.814669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-caa7d39d-0299-45e7-b2c3-43dec546472a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:30.815874 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-caa7d39d-0299-45e7-b2c3-43dec546472a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:30.824600 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1092/2195] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:30.828903 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e264a-b125-4250-9c73-4dbba961d12f None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:30.839255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e264a-b125-4250-9c73-4dbba961d12f None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:30.844251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce54fc-a611-4b97-952c-648cada8c552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:30.844994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce54fc-a611-4b97-952c-648cada8c552 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:30.845378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce54fc-a611-4b97-952c-648cada8c552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:30.845725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce54fc-a611-4b97-952c-648cada8c552 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:30.846563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce54fc-a611-4b97-952c-648cada8c552 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:21:30.846941 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce54fc-a611-4b97-952c-648cada8c552 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:30.848673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce54fc-a611-4b97-952c-648cada8c552 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:30.861590 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1104/2196] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:21:30.865808 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7d39d-0299-45e7-b2c3-43dec546472a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:30.875292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7d39d-0299-45e7-b2c3-43dec546472a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:30.880645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7fe07-2e43-4518-b114-52a548bfbd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:30.881438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7fe07-2e43-4518-b114-52a548bfbd65 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:30.881756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7fe07-2e43-4518-b114-52a548bfbd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:30.882039 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7fe07-2e43-4518-b114-52a548bfbd65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:30.882741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7fe07-2e43-4518-b114-52a548bfbd65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:30.884727 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7fe07-2e43-4518-b114-52a548bfbd65 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:30.918597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1093/2197] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:30 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:30.919667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce54fc-a611-4b97-952c-648cada8c552 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:30.933530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce54fc-a611-4b97-952c-648cada8c552 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:31.014252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236ead7-0f89-4d71-bbf9-277f6f143fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:31.015593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236ead7-0f89-4d71-bbf9-277f6f143fba admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:31.016260 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236ead7-0f89-4d71-bbf9-277f6f143fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:31.016696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236ead7-0f89-4d71-bbf9-277f6f143fba admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:31.017772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236ead7-0f89-4d71-bbf9-277f6f143fba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:31.020334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236ead7-0f89-4d71-bbf9-277f6f143fba admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:31.046392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1105/2198] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:30 2022] POST /identity/v3/users => generated 285 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:21:31.049909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7fe07-2e43-4518-b114-52a548bfbd65 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:31.066867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7fe07-2e43-4518-b114-52a548bfbd65 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:31.071553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70daf80-00b2-4cc2-85e5-1a6c2dda5540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:31.072285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70daf80-00b2-4cc2-85e5-1a6c2dda5540 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:31.072695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70daf80-00b2-4cc2-85e5-1a6c2dda5540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:31.073069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70daf80-00b2-4cc2-85e5-1a6c2dda5540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:31.073924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70daf80-00b2-4cc2-85e5-1a6c2dda5540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:31.075896 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70daf80-00b2-4cc2-85e5-1a6c2dda5540 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:31.087226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1094/2199] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:31.091786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236ead7-0f89-4d71-bbf9-277f6f143fba admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:31.101564 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236ead7-0f89-4d71-bbf9-277f6f143fba admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:31.106214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c267baa-3008-4ae9-90f0-2917eb45478c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:31.106870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c267baa-3008-4ae9-90f0-2917eb45478c admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:31.107158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c267baa-3008-4ae9-90f0-2917eb45478c admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:31.107432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c267baa-3008-4ae9-90f0-2917eb45478c admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:31.108250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c267baa-3008-4ae9-90f0-2917eb45478c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:31.109853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c267baa-3008-4ae9-90f0-2917eb45478c admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:31.126365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1106/2200] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:21:31.130813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70daf80-00b2-4cc2-85e5-1a6c2dda5540 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:31.145989 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70daf80-00b2-4cc2-85e5-1a6c2dda5540 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:31.156400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f5165-40fb-472e-88a2-0cf6cf0f8124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:31.157569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f5165-40fb-472e-88a2-0cf6cf0f8124 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:31.157839 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f5165-40fb-472e-88a2-0cf6cf0f8124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:31.158358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f5165-40fb-472e-88a2-0cf6cf0f8124 admin admin] PATH_INFO: `/v3/projects/8db48ef6a1f44a5db3e1e01d4bb427c4/users/15cd4730159a4e82a93626ae2380f8b0/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:31.181250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f5165-40fb-472e-88a2-0cf6cf0f8124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db48ef6a1f44a5db3e1e01d4bb427c4, user_id=15cd4730159a4e82a93626ae2380f8b0, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:31.186243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f5165-40fb-472e-88a2-0cf6cf0f8124 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:31.208255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1095/2201] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:31 2022] PUT /identity/v3/projects/8db48ef6a1f44a5db3e1e01d4bb427c4/users/15cd4730159a4e82a93626ae2380f8b0/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:31.219192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075084ca-0310-4173-944b-69aabfa73b76 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:31.219535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075084ca-0310-4173-944b-69aabfa73b76 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:31.219809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075084ca-0310-4173-944b-69aabfa73b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:31.264888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-075084ca-0310-4173-944b-69aabfa73b76 None None] MFA Rules not processed for user `15cd4730159a4e82a93626ae2380f8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:31.267924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-075084ca-0310-4173-944b-69aabfa73b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:31.363511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1107/2202] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:31 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:31.376854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f5165-40fb-472e-88a2-0cf6cf0f8124 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:31.378826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-250f5165-40fb-472e-88a2-0cf6cf0f8124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:31.417263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:31.481102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f5165-40fb-472e-88a2-0cf6cf0f8124 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:31.484899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcbd03-46f7-497c-b08e-a07fef4934e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:31.485553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcbd03-46f7-497c-b08e-a07fef4934e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:31.485846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcbd03-46f7-497c-b08e-a07fef4934e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:31.486130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcbd03-46f7-497c-b08e-a07fef4934e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:31.495966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcbd03-46f7-497c-b08e-a07fef4934e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:31.497477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcbd03-46f7-497c-b08e-a07fef4934e7 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:31.506979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1096/2203] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:31.533726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075084ca-0310-4173-944b-69aabfa73b76 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:31.561400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:31.610928 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075084ca-0310-4173-944b-69aabfa73b76 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:31.615977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:31.616954 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:31.617395 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:31.617740 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] PATH_INFO: `/v3/users/15cd4730159a4e82a93626ae2380f8b0` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:31.619010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15cd4730159a4e82a93626ae2380f8b0)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:31.621717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:31.670445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15cd4730159a4e82a93626ae2380f8b0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:31.677662 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15cd4730159a4e82a93626ae2380f8b0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:31.678256 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15cd4730159a4e82a93626ae2380f8b0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:31.690566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15cd4730159a4e82a93626ae2380f8b0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:31.691204 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15cd4730159a4e82a93626ae2380f8b0'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:31.703604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1108/2204] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:31 2022] DELETE /identity/v3/users/15cd4730159a4e82a93626ae2380f8b0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:31.708228 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab9bb1-92d9-423e-8c5f-6b3dd642da90 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:31.708518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab9bb1-92d9-423e-8c5f-6b3dd642da90 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:31.708795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab9bb1-92d9-423e-8c5f-6b3dd642da90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:31.726275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-baab9bb1-92d9-423e-8c5f-6b3dd642da90 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:31.727496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-baab9bb1-92d9-423e-8c5f-6b3dd642da90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:31.784931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1097/2205] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:31.795013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:31.796509 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:31.856720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52a4b-d6fe-4cc1-8de1-5867145b8b26 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:31.860440 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40b727-1ce8-4136-b333-dcccd92e78fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:31.861142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40b727-1ce8-4136-b333-dcccd92e78fb service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:31.861432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40b727-1ce8-4136-b333-dcccd92e78fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:31.861712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40b727-1ce8-4136-b333-dcccd92e78fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:31.872259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40b727-1ce8-4136-b333-dcccd92e78fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:31.873705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40b727-1ce8-4136-b333-dcccd92e78fb service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:31.883126 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1109/2206] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:32.140722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab9bb1-92d9-423e-8c5f-6b3dd642da90 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:32.142902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-baab9bb1-92d9-423e-8c5f-6b3dd642da90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:32.168555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab9bb1-92d9-423e-8c5f-6b3dd642da90 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:32.173790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda7a68-8df7-488f-a822-f5c9c4756b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:32.174786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda7a68-8df7-488f-a822-f5c9c4756b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:32.175410 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda7a68-8df7-488f-a822-f5c9c4756b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:32.175590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda7a68-8df7-488f-a822-f5c9c4756b76 admin admin] PATH_INFO: `/v3/projects/8db48ef6a1f44a5db3e1e01d4bb427c4` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:32.177061 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda7a68-8df7-488f-a822-f5c9c4756b76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db48ef6a1f44a5db3e1e01d4bb427c4)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:32.179200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda7a68-8df7-488f-a822-f5c9c4756b76 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:32.227735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8fda7a68-8df7-488f-a822-f5c9c4756b76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db48ef6a1f44a5db3e1e01d4bb427c4', 'request_id': 'req-8fda7a68-8df7-488f-a822-f5c9c4756b76'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:32.234029 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8fda7a68-8df7-488f-a822-f5c9c4756b76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db48ef6a1f44a5db3e1e01d4bb427c4', 'request_id': 'req-8fda7a68-8df7-488f-a822-f5c9c4756b76'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:32.240110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8fda7a68-8df7-488f-a822-f5c9c4756b76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db48ef6a1f44a5db3e1e01d4bb427c4', 'request_id': 'req-8fda7a68-8df7-488f-a822-f5c9c4756b76'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:32.252248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8fda7a68-8df7-488f-a822-f5c9c4756b76 admin admin] The token cache is being invalidate because project 8db48ef6a1f44a5db3e1e01d4bb427c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:32.252533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8fda7a68-8df7-488f-a822-f5c9c4756b76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:32.253156 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8fda7a68-8df7-488f-a822-f5c9c4756b76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:32.254674 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1098/2207] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:32 2022] DELETE /identity/v3/projects/8db48ef6a1f44a5db3e1e01d4bb427c4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:32.259500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e97d3-7fc0-45eb-acc1-ebb7066eb049 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:32.260656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e97d3-7fc0-45eb-acc1-ebb7066eb049 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:32.261662 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e97d3-7fc0-45eb-acc1-ebb7066eb049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:32.279053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-699e97d3-7fc0-45eb-acc1-ebb7066eb049 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:32.280317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-699e97d3-7fc0-45eb-acc1-ebb7066eb049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:32.320003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:32.344996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1110/2208] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:32.352853 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39281af0-1af2-4d9e-8db8-d5a87191135e None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:32.353154 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39281af0-1af2-4d9e-8db8-d5a87191135e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:32.353439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39281af0-1af2-4d9e-8db8-d5a87191135e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:32.373456 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-39281af0-1af2-4d9e-8db8-d5a87191135e None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:32.374495 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-39281af0-1af2-4d9e-8db8-d5a87191135e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:32.383102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1099/2209] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:32.392692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e97d3-7fc0-45eb-acc1-ebb7066eb049 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:32.403354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e97d3-7fc0-45eb-acc1-ebb7066eb049 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:32.410091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f48586-9aca-4a23-8083-14245f17a7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:32.411294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f48586-9aca-4a23-8083-14245f17a7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:32.412190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f48586-9aca-4a23-8083-14245f17a7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:32.413217 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f48586-9aca-4a23-8083-14245f17a7a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:32.414200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f48586-9aca-4a23-8083-14245f17a7a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:21:32.414521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f48586-9aca-4a23-8083-14245f17a7a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:32.416274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f48586-9aca-4a23-8083-14245f17a7a0 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:32.432698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1111/2210] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:21:32.435063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39281af0-1af2-4d9e-8db8-d5a87191135e None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:32.444280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39281af0-1af2-4d9e-8db8-d5a87191135e None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:32.450009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131f9aa-34a8-40e9-b15c-e926782d3f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:32.450673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131f9aa-34a8-40e9-b15c-e926782d3f07 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:32.451027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131f9aa-34a8-40e9-b15c-e926782d3f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:32.451262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131f9aa-34a8-40e9-b15c-e926782d3f07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:32.452002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131f9aa-34a8-40e9-b15c-e926782d3f07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:32.454300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131f9aa-34a8-40e9-b15c-e926782d3f07 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:32.485689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1100/2211] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:32 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:32.492775 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f48586-9aca-4a23-8083-14245f17a7a0 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:32.504051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f48586-9aca-4a23-8083-14245f17a7a0 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:32.576968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbbf1c-c42b-43c7-b039-144a9eb07555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:32.577704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbbf1c-c42b-43c7-b039-144a9eb07555 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:32.578011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbbf1c-c42b-43c7-b039-144a9eb07555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:32.578272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbbf1c-c42b-43c7-b039-144a9eb07555 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:32.579020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbbf1c-c42b-43c7-b039-144a9eb07555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:32.581216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbbf1c-c42b-43c7-b039-144a9eb07555 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:32.605650 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1112/2212] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:32 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:32.609563 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131f9aa-34a8-40e9-b15c-e926782d3f07 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:32.620440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131f9aa-34a8-40e9-b15c-e926782d3f07 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:32.626218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397504d1-3f44-453e-a335-94399818be70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:32.627032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397504d1-3f44-453e-a335-94399818be70 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:32.627322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397504d1-3f44-453e-a335-94399818be70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:32.627624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397504d1-3f44-453e-a335-94399818be70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:32.628401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397504d1-3f44-453e-a335-94399818be70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:32.630098 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397504d1-3f44-453e-a335-94399818be70 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:32.652520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1101/2213] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:32.657027 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbbf1c-c42b-43c7-b039-144a9eb07555 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:32.668989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbbf1c-c42b-43c7-b039-144a9eb07555 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:32.673795 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b5dac-31bb-4730-89ab-bf562d641eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:32.674472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b5dac-31bb-4730-89ab-bf562d641eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:32.674762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b5dac-31bb-4730-89ab-bf562d641eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:32.675043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b5dac-31bb-4730-89ab-bf562d641eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:32.675772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b5dac-31bb-4730-89ab-bf562d641eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:32.678906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b5dac-31bb-4730-89ab-bf562d641eb8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:32.701579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1113/2214] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:32.703419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397504d1-3f44-453e-a335-94399818be70 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:32.716013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397504d1-3f44-453e-a335-94399818be70 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:32.721441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466d153-1050-4ecf-a234-024e7f2f5073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:32.722336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466d153-1050-4ecf-a234-024e7f2f5073 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:32.722625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466d153-1050-4ecf-a234-024e7f2f5073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:32.722908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466d153-1050-4ecf-a234-024e7f2f5073 admin admin] PATH_INFO: `/v3/projects/72db86ca9d3a45b49c7bb140f6b0fa10/users/dd94a883b0cb4dc4bcc57c221577b29a/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:32.727113 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:32.744253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466d153-1050-4ecf-a234-024e7f2f5073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72db86ca9d3a45b49c7bb140f6b0fa10, user_id=dd94a883b0cb4dc4bcc57c221577b29a, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:32.747217 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466d153-1050-4ecf-a234-024e7f2f5073 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:32.768144 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1102/2215] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:32 2022] PUT /identity/v3/projects/72db86ca9d3a45b49c7bb140f6b0fa10/users/dd94a883b0cb4dc4bcc57c221577b29a/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:32.774632 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c95384-ca57-4bd0-a058-82efaa786b7b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:32.774929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c95384-ca57-4bd0-a058-82efaa786b7b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:32.775207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c95384-ca57-4bd0-a058-82efaa786b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:32.808055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-f3c95384-ca57-4bd0-a058-82efaa786b7b None None] MFA Rules not processed for user `dd94a883b0cb4dc4bcc57c221577b29a`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:32.809375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f3c95384-ca57-4bd0-a058-82efaa786b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:32.862955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466d153-1050-4ecf-a234-024e7f2f5073 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:32.864748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-5466d153-1050-4ecf-a234-024e7f2f5073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:32.884576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1114/2216] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:32.895454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c95384-ca57-4bd0-a058-82efaa786b7b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:32.896784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f3c95384-ca57-4bd0-a058-82efaa786b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:32.932080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466d153-1050-4ecf-a234-024e7f2f5073 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:32.936744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:32.937619 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:32.937906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:32.938190 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] PATH_INFO: `/v3/users/3d39e43cb0964fdd920a10ccb6249d29` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:32.938354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:32.939387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d39e43cb0964fdd920a10ccb6249d29)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:32.941485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:32.954847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c95384-ca57-4bd0-a058-82efaa786b7b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:32.958486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4015a53-6b2a-4f35-a864-1bd8a8dafcfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:32.959131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4015a53-6b2a-4f35-a864-1bd8a8dafcfb service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:32.959415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4015a53-6b2a-4f35-a864-1bd8a8dafcfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:32.959694 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4015a53-6b2a-4f35-a864-1bd8a8dafcfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:32.968794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4015a53-6b2a-4f35-a864-1bd8a8dafcfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:32.970301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4015a53-6b2a-4f35-a864-1bd8a8dafcfb service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:32.979567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1115/2217] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:21:32.988758 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d39e43cb0964fdd920a10ccb6249d29'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:32.989474 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d39e43cb0964fdd920a10ccb6249d29'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:32.998497 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d39e43cb0964fdd920a10ccb6249d29'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.011291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d39e43cb0964fdd920a10ccb6249d29'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.011951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d39e43cb0964fdd920a10ccb6249d29'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.026093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1103/2218] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:32 2022] DELETE /identity/v3/users/3d39e43cb0964fdd920a10ccb6249d29 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:33.030613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e89c7-b136-4041-9c03-46a31342d354 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:33.031015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e89c7-b136-4041-9c03-46a31342d354 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:33.031306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e89c7-b136-4041-9c03-46a31342d354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:33.049216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-551e89c7-b136-4041-9c03-46a31342d354 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:33.050437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-551e89c7-b136-4041-9c03-46a31342d354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:33.112112 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1116/2219] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:33.123343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:33.125002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:33.199549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966e82f-dabd-42be-9544-97054b670e03 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:33.205684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6d7a5-8781-4ebd-918b-9e17eaed16d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:33.206343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6d7a5-8781-4ebd-918b-9e17eaed16d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:33.206730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6d7a5-8781-4ebd-918b-9e17eaed16d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:33.207028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6d7a5-8781-4ebd-918b-9e17eaed16d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:33.220397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6d7a5-8781-4ebd-918b-9e17eaed16d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:33.222847 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6d7a5-8781-4ebd-918b-9e17eaed16d9 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:33.236696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e89c7-b136-4041-9c03-46a31342d354 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:33.238158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1104/2220] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:21:33.238284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-551e89c7-b136-4041-9c03-46a31342d354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:33.254981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e89c7-b136-4041-9c03-46a31342d354 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:33.264754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:33.265782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:33.266073 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:33.266429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] PATH_INFO: `/v3/users/bcf5b641ef544274a7722607461fca20` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:33.278662 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcf5b641ef544274a7722607461fca20)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:33.282886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:33.349293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcf5b641ef544274a7722607461fca20'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.360502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcf5b641ef544274a7722607461fca20'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.361208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcf5b641ef544274a7722607461fca20'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.377997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcf5b641ef544274a7722607461fca20'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.378979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcf5b641ef544274a7722607461fca20'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.399088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1117/2221] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:33 2022] DELETE /identity/v3/users/bcf5b641ef544274a7722607461fca20 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:33.404416 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c20e8-37d6-4529-95c7-f807e3537f9d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:33.404714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c20e8-37d6-4529-95c7-f807e3537f9d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:33.404998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c20e8-37d6-4529-95c7-f807e3537f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:33.418947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:33.425500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-962c20e8-37d6-4529-95c7-f807e3537f9d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:33.428026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-962c20e8-37d6-4529-95c7-f807e3537f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:33.492138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1105/2222] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:33.502559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:33.503965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:33.567552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea51da-491b-4121-8ab0-4b7ec5529332 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:33.571237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd661d5-e5bb-4ade-9be7-b248db9d383a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:33.571888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd661d5-e5bb-4ade-9be7-b248db9d383a service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:33.572255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd661d5-e5bb-4ade-9be7-b248db9d383a service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:33.572577 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd661d5-e5bb-4ade-9be7-b248db9d383a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:33.580955 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:33.583393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd661d5-e5bb-4ade-9be7-b248db9d383a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:33.585478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd661d5-e5bb-4ade-9be7-b248db9d383a service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:33.597708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1118/2223] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:33.697053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c20e8-37d6-4529-95c7-f807e3537f9d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:33.699180 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-962c20e8-37d6-4529-95c7-f807e3537f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:33.710916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c20e8-37d6-4529-95c7-f807e3537f9d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:33.716083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:33.716953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:33.717237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:33.717514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] PATH_INFO: `/v3/projects/fe1f21ceb43f4fce9bf102ed7fbfb709` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:33.718712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe1f21ceb43f4fce9bf102ed7fbfb709)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:33.720804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:33.774986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe1f21ceb43f4fce9bf102ed7fbfb709', 'request_id': 'req-f0947845-d3bb-4122-80b1-ddfe3eedb924'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.781484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe1f21ceb43f4fce9bf102ed7fbfb709', 'request_id': 'req-f0947845-d3bb-4122-80b1-ddfe3eedb924'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.788109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe1f21ceb43f4fce9bf102ed7fbfb709', 'request_id': 'req-f0947845-d3bb-4122-80b1-ddfe3eedb924'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.799451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] The token cache is being invalidate because project fe1f21ceb43f4fce9bf102ed7fbfb709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:33.799810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.800572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.805143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1106/2224] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:33 2022] DELETE /identity/v3/projects/fe1f21ceb43f4fce9bf102ed7fbfb709 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:33.806405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd661d5-e5bb-4ade-9be7-b248db9d383a service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:33.817417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd661d5-e5bb-4ade-9be7-b248db9d383a service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:33.893785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a08246-f554-4721-824b-38c16d5ae5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:33.894695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a08246-f554-4721-824b-38c16d5ae5ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:33.894976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a08246-f554-4721-824b-38c16d5ae5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:33.895271 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a08246-f554-4721-824b-38c16d5ae5ed admin admin] PATH_INFO: `/v3/users/542da5064827434c93cf287a33f80ff9` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:33.896608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a08246-f554-4721-824b-38c16d5ae5ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=542da5064827434c93cf287a33f80ff9)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:33.898651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a08246-f554-4721-824b-38c16d5ae5ed admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:33.958031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-40a08246-f554-4721-824b-38c16d5ae5ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '542da5064827434c93cf287a33f80ff9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.969018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-40a08246-f554-4721-824b-38c16d5ae5ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '542da5064827434c93cf287a33f80ff9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.969856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-40a08246-f554-4721-824b-38c16d5ae5ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '542da5064827434c93cf287a33f80ff9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.990827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-40a08246-f554-4721-824b-38c16d5ae5ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '542da5064827434c93cf287a33f80ff9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:33.991824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-40a08246-f554-4721-824b-38c16d5ae5ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '542da5064827434c93cf287a33f80ff9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:34.011663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1119/2225] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:33 2022] DELETE /identity/v3/users/542da5064827434c93cf287a33f80ff9 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:34.062264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.063819 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:34.088779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0947845-d3bb-4122-80b1-ddfe3eedb924 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:34.154037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc10eef-ff42-4a1f-9fed-310f99afcd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:34.154907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc10eef-ff42-4a1f-9fed-310f99afcd61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.155191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc10eef-ff42-4a1f-9fed-310f99afcd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.155493 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc10eef-ff42-4a1f-9fed-310f99afcd61 admin admin] PATH_INFO: `/v3/projects/564c2fe9f88748a2825a129fe97f7181` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.156688 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc10eef-ff42-4a1f-9fed-310f99afcd61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=564c2fe9f88748a2825a129fe97f7181)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:34.158601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc10eef-ff42-4a1f-9fed-310f99afcd61 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:34.195597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:34.209172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-5bc10eef-ff42-4a1f-9fed-310f99afcd61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '564c2fe9f88748a2825a129fe97f7181', 'request_id': 'req-5bc10eef-ff42-4a1f-9fed-310f99afcd61'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:34.216978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-5bc10eef-ff42-4a1f-9fed-310f99afcd61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '564c2fe9f88748a2825a129fe97f7181', 'request_id': 'req-5bc10eef-ff42-4a1f-9fed-310f99afcd61'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:34.224489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-5bc10eef-ff42-4a1f-9fed-310f99afcd61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '564c2fe9f88748a2825a129fe97f7181', 'request_id': 'req-5bc10eef-ff42-4a1f-9fed-310f99afcd61'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:34.236741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-5bc10eef-ff42-4a1f-9fed-310f99afcd61 admin admin] The token cache is being invalidate because project 564c2fe9f88748a2825a129fe97f7181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:34.237037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-5bc10eef-ff42-4a1f-9fed-310f99afcd61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:34.237687 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-5bc10eef-ff42-4a1f-9fed-310f99afcd61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:34.239574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1107/2226] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:34 2022] DELETE /identity/v3/projects/564c2fe9f88748a2825a129fe97f7181 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:34.245213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304fd54-8ba4-44c9-8f0f-eae3e8822ed3 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.245617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304fd54-8ba4-44c9-8f0f-eae3e8822ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.245778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304fd54-8ba4-44c9-8f0f-eae3e8822ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.273646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-5304fd54-8ba4-44c9-8f0f-eae3e8822ed3 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:34.275032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5304fd54-8ba4-44c9-8f0f-eae3e8822ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:34.337467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1120/2227] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:34.346624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4cbea-a0f9-4b6a-acc2-954deedfbbcf None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.347383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4cbea-a0f9-4b6a-acc2-954deedfbbcf None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.347383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4cbea-a0f9-4b6a-acc2-954deedfbbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.368096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-e0a4cbea-a0f9-4b6a-acc2-954deedfbbcf None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:34.369069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e0a4cbea-a0f9-4b6a-acc2-954deedfbbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:34.381368 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1108/2228] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:34.390447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304fd54-8ba4-44c9-8f0f-eae3e8822ed3 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.397334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304fd54-8ba4-44c9-8f0f-eae3e8822ed3 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:34.402535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733434de-d305-4c7f-976d-b4add87b6b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:34.403280 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733434de-d305-4c7f-976d-b4add87b6b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.403557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733434de-d305-4c7f-976d-b4add87b6b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.406360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733434de-d305-4c7f-976d-b4add87b6b3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.406360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733434de-d305-4c7f-976d-b4add87b6b3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:21:34.406703 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733434de-d305-4c7f-976d-b4add87b6b3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:34.407582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4cbea-a0f9-4b6a-acc2-954deedfbbcf None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.408611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733434de-d305-4c7f-976d-b4add87b6b3b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:34.409117 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e0a4cbea-a0f9-4b6a-acc2-954deedfbbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:34.425493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1121/2229] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:21:34.429378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733434de-d305-4c7f-976d-b4add87b6b3b admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.433413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4cbea-a0f9-4b6a-acc2-954deedfbbcf None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:34.452161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733434de-d305-4c7f-976d-b4add87b6b3b admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:34.452161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:34.452522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:34.452522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.452522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.452522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] PATH_INFO: `/v3/projects/43bc5e3de097424789b4da0d68d040a1` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.452896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce69e1-9f94-43a2-8221-d83da3f64c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:34.452896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce69e1-9f94-43a2-8221-d83da3f64c8b admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.453177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce69e1-9f94-43a2-8221-d83da3f64c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.453816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce69e1-9f94-43a2-8221-d83da3f64c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.453909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] RBAC: Authorizing `identity:delete_project(project_id=43bc5e3de097424789b4da0d68d040a1)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:34.454660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce69e1-9f94-43a2-8221-d83da3f64c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:34.457668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce69e1-9f94-43a2-8221-d83da3f64c8b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:34.458391 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:34.491303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1122/2230] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:34 2022] POST /identity/v3/projects => generated 391 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:34.491303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24960e-9dbc-4b13-8bba-2b51b17543dc None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.491303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24960e-9dbc-4b13-8bba-2b51b17543dc None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.491303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24960e-9dbc-4b13-8bba-2b51b17543dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.508289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-bb24960e-9dbc-4b13-8bba-2b51b17543dc None None] MFA Rules not processed for user `dd94a883b0cb4dc4bcc57c221577b29a`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:34.509857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-bb24960e-9dbc-4b13-8bba-2b51b17543dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:34.515200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43bc5e3de097424789b4da0d68d040a1', 'request_id': 'req-9a347a4d-2bc6-4b43-8292-ab083a74060e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:34.524700 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43bc5e3de097424789b4da0d68d040a1', 'request_id': 'req-9a347a4d-2bc6-4b43-8292-ab083a74060e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:34.532746 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43bc5e3de097424789b4da0d68d040a1', 'request_id': 'req-9a347a4d-2bc6-4b43-8292-ab083a74060e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:34.549300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] The token cache is being invalidate because project 43bc5e3de097424789b4da0d68d040a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:34.549848 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:34.550557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:34.552379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1109/2231] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:34 2022] DELETE /identity/v3/projects/43bc5e3de097424789b4da0d68d040a1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:34.553140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.557304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:34.582925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1123/2232] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:34.584860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9911694-7cb9-4908-8775-fffcd4d37d20 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.585155 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9911694-7cb9-4908-8775-fffcd4d37d20 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.585494 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9911694-7cb9-4908-8775-fffcd4d37d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.608092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-f9911694-7cb9-4908-8775-fffcd4d37d20 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:34.609371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f9911694-7cb9-4908-8775-fffcd4d37d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:34.629238 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a347a4d-2bc6-4b43-8292-ab083a74060e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:34.636130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c94a31-d4e9-45eb-8a78-4ca4a39bb4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:34.636879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c94a31-d4e9-45eb-8a78-4ca4a39bb4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.637164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c94a31-d4e9-45eb-8a78-4ca4a39bb4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.637448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c94a31-d4e9-45eb-8a78-4ca4a39bb4c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.638197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c94a31-d4e9-45eb-8a78-4ca4a39bb4c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:34.640097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c94a31-d4e9-45eb-8a78-4ca4a39bb4c7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:34.668599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1110/2233] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:34 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:34.672447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c94a31-d4e9-45eb-8a78-4ca4a39bb4c7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.684115 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c94a31-d4e9-45eb-8a78-4ca4a39bb4c7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:34.689382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1463f-0be6-4c92-8124-1bb771b3dc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:34.690062 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1463f-0be6-4c92-8124-1bb771b3dc86 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.690351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1463f-0be6-4c92-8124-1bb771b3dc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.690632 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1463f-0be6-4c92-8124-1bb771b3dc86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.691361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1463f-0be6-4c92-8124-1bb771b3dc86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:34.698479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1463f-0be6-4c92-8124-1bb771b3dc86 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:34.709137 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1124/2234] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:34.711285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5194cea-93e2-4a07-ad59-97421f473a05 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.711589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5194cea-93e2-4a07-ad59-97421f473a05 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.711920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5194cea-93e2-4a07-ad59-97421f473a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.724551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1111/2235] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:34.728450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1463f-0be6-4c92-8124-1bb771b3dc86 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.738665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1463f-0be6-4c92-8124-1bb771b3dc86 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:34.739373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-f5194cea-93e2-4a07-ad59-97421f473a05 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:34.740648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f5194cea-93e2-4a07-ad59-97421f473a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:34.745529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424014cf-98c9-43ef-9989-432743fce887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:34.746197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424014cf-98c9-43ef-9989-432743fce887 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.746477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424014cf-98c9-43ef-9989-432743fce887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.746749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424014cf-98c9-43ef-9989-432743fce887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.747525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424014cf-98c9-43ef-9989-432743fce887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:34.749083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424014cf-98c9-43ef-9989-432743fce887 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:34.749869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1125/2236] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:34.754454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5194cea-93e2-4a07-ad59-97421f473a05 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.763679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5194cea-93e2-4a07-ad59-97421f473a05 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:34.764773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1112/2237] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:21:34.768622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b368f8-cc59-4bfd-b9fb-b4efd7e59d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:34.768762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424014cf-98c9-43ef-9989-432743fce887 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.769406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b368f8-cc59-4bfd-b9fb-b4efd7e59d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.769693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b368f8-cc59-4bfd-b9fb-b4efd7e59d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.769971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b368f8-cc59-4bfd-b9fb-b4efd7e59d1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.770700 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b368f8-cc59-4bfd-b9fb-b4efd7e59d1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:21:34.771010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b368f8-cc59-4bfd-b9fb-b4efd7e59d1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:34.772631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b368f8-cc59-4bfd-b9fb-b4efd7e59d1b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:34.777766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:34.781919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424014cf-98c9-43ef-9989-432743fce887 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:34.786399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1126/2238] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:21:34.791666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b368f8-cc59-4bfd-b9fb-b4efd7e59d1b admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.793018 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932076cb-4eec-40fa-a26f-6bb3e7510595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:34.793915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932076cb-4eec-40fa-a26f-6bb3e7510595 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.794200 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932076cb-4eec-40fa-a26f-6bb3e7510595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.794475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932076cb-4eec-40fa-a26f-6bb3e7510595 admin admin] PATH_INFO: `/v3/projects/59c145a95ed94a538bafaf99e7429607/users/31ad277d407e4d07afa20304a4bf2c5f/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.803253 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b368f8-cc59-4bfd-b9fb-b4efd7e59d1b admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:34.810221 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68753608-5947-4886-9965-988c8a468e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:34.810446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932076cb-4eec-40fa-a26f-6bb3e7510595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c145a95ed94a538bafaf99e7429607, user_id=31ad277d407e4d07afa20304a4bf2c5f, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:34.810897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68753608-5947-4886-9965-988c8a468e21 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.811184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68753608-5947-4886-9965-988c8a468e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.811470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68753608-5947-4886-9965-988c8a468e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.812371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68753608-5947-4886-9965-988c8a468e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:34.813214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932076cb-4eec-40fa-a26f-6bb3e7510595 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:34.814284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68753608-5947-4886-9965-988c8a468e21 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:34.840207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1113/2239] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:34 2022] PUT /identity/v3/projects/59c145a95ed94a538bafaf99e7429607/users/31ad277d407e4d07afa20304a4bf2c5f/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:34.843250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53349cff-ae24-4c7b-ac51-17caa5b7a692 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.845316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53349cff-ae24-4c7b-ac51-17caa5b7a692 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.845667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53349cff-ae24-4c7b-ac51-17caa5b7a692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.865279 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1127/2240] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:34 2022] POST /identity/v3/projects => generated 387 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:21:34.866537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68753608-5947-4886-9965-988c8a468e21 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.878451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-53349cff-ae24-4c7b-ac51-17caa5b7a692 None None] MFA Rules not processed for user `31ad277d407e4d07afa20304a4bf2c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:34.879850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-53349cff-ae24-4c7b-ac51-17caa5b7a692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:34.924172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:34.942838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68753608-5947-4886-9965-988c8a468e21 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:34.947693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11382bec-b682-4cff-9ecd-eb2a1a67ff5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:34.948409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11382bec-b682-4cff-9ecd-eb2a1a67ff5a admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.948699 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11382bec-b682-4cff-9ecd-eb2a1a67ff5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.948984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11382bec-b682-4cff-9ecd-eb2a1a67ff5a admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.949676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11382bec-b682-4cff-9ecd-eb2a1a67ff5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:34.951645 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11382bec-b682-4cff-9ecd-eb2a1a67ff5a admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:34.954608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1114/2241] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:34.965744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53349cff-ae24-4c7b-ac51-17caa5b7a692 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.967646 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-53349cff-ae24-4c7b-ac51-17caa5b7a692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:34.976354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1128/2242] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:34 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:34.979779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11382bec-b682-4cff-9ecd-eb2a1a67ff5a admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:34.989733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11382bec-b682-4cff-9ecd-eb2a1a67ff5a admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:34.996345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cacb7-cc2b-4c14-b1c5-d48c59f5eccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:34.997219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cacb7-cc2b-4c14-b1c5-d48c59f5eccb admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:34.997651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cacb7-cc2b-4c14-b1c5-d48c59f5eccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:34.998161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cacb7-cc2b-4c14-b1c5-d48c59f5eccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:34.999044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cacb7-cc2b-4c14-b1c5-d48c59f5eccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:35.001186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cacb7-cc2b-4c14-b1c5-d48c59f5eccb admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:35.013908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1129/2243] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:35.018029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cacb7-cc2b-4c14-b1c5-d48c59f5eccb admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:35.029658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cacb7-cc2b-4c14-b1c5-d48c59f5eccb admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:35.037785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb349ed-4189-4ca9-a1d4-fa6ada0ebd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:35.038522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb349ed-4189-4ca9-a1d4-fa6ada0ebd04 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:35.038817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb349ed-4189-4ca9-a1d4-fa6ada0ebd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:35.039099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb349ed-4189-4ca9-a1d4-fa6ada0ebd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:35.039835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb349ed-4189-4ca9-a1d4-fa6ada0ebd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:35.042238 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb349ed-4189-4ca9-a1d4-fa6ada0ebd04 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:35.044559 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53349cff-ae24-4c7b-ac51-17caa5b7a692 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:35.049308 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fc362-1b5b-4f24-ae14-a71d86b25707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:35.049965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fc362-1b5b-4f24-ae14-a71d86b25707 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:35.050253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fc362-1b5b-4f24-ae14-a71d86b25707 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:35.050594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fc362-1b5b-4f24-ae14-a71d86b25707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:35.057601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1130/2244] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:35.058690 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb349ed-4189-4ca9-a1d4-fa6ada0ebd04 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:35.060539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fc362-1b5b-4f24-ae14-a71d86b25707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:35.061937 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fc362-1b5b-4f24-ae14-a71d86b25707 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:35.070319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb349ed-4189-4ca9-a1d4-fa6ada0ebd04 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:35.075922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767a35d-a6c6-437d-9cf2-7f27d71ae5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:35.076536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767a35d-a6c6-437d-9cf2-7f27d71ae5c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:35.076827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767a35d-a6c6-437d-9cf2-7f27d71ae5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:35.077111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767a35d-a6c6-437d-9cf2-7f27d71ae5c7 admin admin] PATH_INFO: `/v3/projects/8659de13aa20446c923054374a0f67fe/users/2751c41434d54bcfab4efbd8975af783/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:35.084642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1115/2245] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:21:35.094466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767a35d-a6c6-437d-9cf2-7f27d71ae5c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8659de13aa20446c923054374a0f67fe, user_id=2751c41434d54bcfab4efbd8975af783, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:35.097604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767a35d-a6c6-437d-9cf2-7f27d71ae5c7 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:35.144423 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1131/2246] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:35 2022] PUT /identity/v3/projects/8659de13aa20446c923054374a0f67fe/users/2751c41434d54bcfab4efbd8975af783/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:35.147944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db6ec3-85b6-4100-94a6-32948b7c94a5 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:35.148373 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db6ec3-85b6-4100-94a6-32948b7c94a5 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:35.148854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db6ec3-85b6-4100-94a6-32948b7c94a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:35.181313 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-d3db6ec3-85b6-4100-94a6-32948b7c94a5 None None] MFA Rules not processed for user `2751c41434d54bcfab4efbd8975af783`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:35.182837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d3db6ec3-85b6-4100-94a6-32948b7c94a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:35.265909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1116/2247] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:35.272768 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767a35d-a6c6-437d-9cf2-7f27d71ae5c7 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:35.369114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767a35d-a6c6-437d-9cf2-7f27d71ae5c7 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:35.376525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347989c8-edae-4c11-a30f-2b264351b941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:35.377232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347989c8-edae-4c11-a30f-2b264351b941 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:35.377547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347989c8-edae-4c11-a30f-2b264351b941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:35.377831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347989c8-edae-4c11-a30f-2b264351b941 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:35.378623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347989c8-edae-4c11-a30f-2b264351b941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:35.380753 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347989c8-edae-4c11-a30f-2b264351b941 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:35.421958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1132/2248] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:35 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:35.426571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db6ec3-85b6-4100-94a6-32948b7c94a5 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:35.433842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:35.441051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db6ec3-85b6-4100-94a6-32948b7c94a5 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:35.548951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21946ba8-a310-450d-9313-d1cc9fdd48be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:35.549854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21946ba8-a310-450d-9313-d1cc9fdd48be admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:35.550292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21946ba8-a310-450d-9313-d1cc9fdd48be admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:35.550718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21946ba8-a310-450d-9313-d1cc9fdd48be admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:35.551677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21946ba8-a310-450d-9313-d1cc9fdd48be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:35.553828 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21946ba8-a310-450d-9313-d1cc9fdd48be admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:35.611078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1117/2249] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:35 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:35.611194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347989c8-edae-4c11-a30f-2b264351b941 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:35.626318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347989c8-edae-4c11-a30f-2b264351b941 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:35.649251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf5862-140a-4c12-bf64-89bfb3659972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:35.650611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf5862-140a-4c12-bf64-89bfb3659972 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:35.650900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf5862-140a-4c12-bf64-89bfb3659972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:35.651186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf5862-140a-4c12-bf64-89bfb3659972 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:35.652002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf5862-140a-4c12-bf64-89bfb3659972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:35.655105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf5862-140a-4c12-bf64-89bfb3659972 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:35.700305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1133/2250] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:35.702527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21946ba8-a310-450d-9313-d1cc9fdd48be admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:35.717289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21946ba8-a310-450d-9313-d1cc9fdd48be admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:35.740346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11cbb0-69cf-46f9-a033-9e58d88c89e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:35.741324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11cbb0-69cf-46f9-a033-9e58d88c89e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:35.741617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11cbb0-69cf-46f9-a033-9e58d88c89e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:35.741908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11cbb0-69cf-46f9-a033-9e58d88c89e6 admin admin] PATH_INFO: `/v3/projects/aae329fd3e004af7aeb2e4bf77845c74/users/9e6403d8c2f5412099d011dc8764a99c/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:35.782178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11cbb0-69cf-46f9-a033-9e58d88c89e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae329fd3e004af7aeb2e4bf77845c74, user_id=9e6403d8c2f5412099d011dc8764a99c, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:35.786859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11cbb0-69cf-46f9-a033-9e58d88c89e6 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:35.824396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1118/2251] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:35 2022] PUT /identity/v3/projects/aae329fd3e004af7aeb2e4bf77845c74/users/9e6403d8c2f5412099d011dc8764a99c/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:35.824906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf5862-140a-4c12-bf64-89bfb3659972 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:35.837679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf5862-140a-4c12-bf64-89bfb3659972 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:35.911725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e5c54-2cb7-422e-a246-06abe4a97ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:35.913325 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e5c54-2cb7-422e-a246-06abe4a97ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:35.913854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e5c54-2cb7-422e-a246-06abe4a97ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:35.914367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e5c54-2cb7-422e-a246-06abe4a97ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:35.915532 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e5c54-2cb7-422e-a246-06abe4a97ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:35.918230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e5c54-2cb7-422e-a246-06abe4a97ee7 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:35.921311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:35.936363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1134/2252] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:21:35.938104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11cbb0-69cf-46f9-a033-9e58d88c89e6 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:35.951699 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11cbb0-69cf-46f9-a033-9e58d88c89e6 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:35.959775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f124c-8a45-4243-861c-4d0738bc5d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:35.959775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f124c-8a45-4243-861c-4d0738bc5d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:35.959775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f124c-8a45-4243-861c-4d0738bc5d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:35.960783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f124c-8a45-4243-861c-4d0738bc5d45 admin admin] PATH_INFO: `/v3/projects/aae329fd3e004af7aeb2e4bf77845c74/users/9e6403d8c2f5412099d011dc8764a99c/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:35.963019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f124c-8a45-4243-861c-4d0738bc5d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae329fd3e004af7aeb2e4bf77845c74, user_id=9e6403d8c2f5412099d011dc8764a99c, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:35.966019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f124c-8a45-4243-861c-4d0738bc5d45 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:35.978126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1119/2253] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:35 2022] PUT /identity/v3/projects/aae329fd3e004af7aeb2e4bf77845c74/users/9e6403d8c2f5412099d011dc8764a99c/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:35.989158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052514f-ac25-44be-b3b5-f7184915ea76 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:35.989339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052514f-ac25-44be-b3b5-f7184915ea76 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:35.989619 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052514f-ac25-44be-b3b5-f7184915ea76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:36.028502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-7052514f-ac25-44be-b3b5-f7184915ea76 None None] MFA Rules not processed for user `9e6403d8c2f5412099d011dc8764a99c`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:36.029807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7052514f-ac25-44be-b3b5-f7184915ea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:36.135774 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1135/2254] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:36.148898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f124c-8a45-4243-861c-4d0738bc5d45 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:36.213531 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f124c-8a45-4243-861c-4d0738bc5d45 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:36.217601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a7a04-8df3-49a2-9415-629dc9cc065f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:36.218735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a7a04-8df3-49a2-9415-629dc9cc065f service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:36.219032 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a7a04-8df3-49a2-9415-629dc9cc065f service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:36.219318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a7a04-8df3-49a2-9415-629dc9cc065f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:36.231626 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a7a04-8df3-49a2-9415-629dc9cc065f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:36.233398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a7a04-8df3-49a2-9415-629dc9cc065f service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:36.246127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1120/2255] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:36.823649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052514f-ac25-44be-b3b5-f7184915ea76 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:36.905957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052514f-ac25-44be-b3b5-f7184915ea76 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:36.912204 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:36.913742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:36.914029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:36.914363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] PATH_INFO: `/v3/users/2751c41434d54bcfab4efbd8975af783` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:36.916533 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2751c41434d54bcfab4efbd8975af783)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:36.917759 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:36.972960 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2751c41434d54bcfab4efbd8975af783'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:36.981445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2751c41434d54bcfab4efbd8975af783'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:36.982065 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2751c41434d54bcfab4efbd8975af783'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:36.993920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2751c41434d54bcfab4efbd8975af783'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:36.995087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2751c41434d54bcfab4efbd8975af783'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:37.006989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1136/2256] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:36 2022] DELETE /identity/v3/users/2751c41434d54bcfab4efbd8975af783 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:37.011934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bee35c-3480-46cd-8e2e-17e75cd641f3 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:37.012443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bee35c-3480-46cd-8e2e-17e75cd641f3 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:37.012531 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bee35c-3480-46cd-8e2e-17e75cd641f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:37.029849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-e2bee35c-3480-46cd-8e2e-17e75cd641f3 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:37.031065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e2bee35c-3480-46cd-8e2e-17e75cd641f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:37.080231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:37.089985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1121/2257] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:21:37.099429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:37.100868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:37.106660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:37.160852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0b92c-3dea-46aa-a3e1-f3349436d583 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:37.164654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce3cb4-0e00-4c9e-ba27-fd01b7ee6146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:37.165294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce3cb4-0e00-4c9e-ba27-fd01b7ee6146 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:37.165575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce3cb4-0e00-4c9e-ba27-fd01b7ee6146 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:37.165849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce3cb4-0e00-4c9e-ba27-fd01b7ee6146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:37.175051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce3cb4-0e00-4c9e-ba27-fd01b7ee6146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:37.176517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce3cb4-0e00-4c9e-ba27-fd01b7ee6146 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:37.186252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1137/2258] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:37.453861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bee35c-3480-46cd-8e2e-17e75cd641f3 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:37.455574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e2bee35c-3480-46cd-8e2e-17e75cd641f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:37.474690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bee35c-3480-46cd-8e2e-17e75cd641f3 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:37.480384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:37.481261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:37.481543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:37.481846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] PATH_INFO: `/v3/projects/8659de13aa20446c923054374a0f67fe` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:37.483034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8659de13aa20446c923054374a0f67fe)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:37.485150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:37.534260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8659de13aa20446c923054374a0f67fe', 'request_id': 'req-9049b462-9dfb-4e70-8376-5c9ec2add896'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:37.541353 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8659de13aa20446c923054374a0f67fe', 'request_id': 'req-9049b462-9dfb-4e70-8376-5c9ec2add896'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:37.547931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8659de13aa20446c923054374a0f67fe', 'request_id': 'req-9049b462-9dfb-4e70-8376-5c9ec2add896'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:37.557901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] The token cache is being invalidate because project 8659de13aa20446c923054374a0f67fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:37.558185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:37.558828 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:37.560385 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1122/2259] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:37 2022] DELETE /identity/v3/projects/8659de13aa20446c923054374a0f67fe => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:37.564289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce3cb4-0e00-4c9e-ba27-fd01b7ee6146 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:37.573816 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce3cb4-0e00-4c9e-ba27-fd01b7ee6146 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:37.628617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd6826-f06f-45ea-83e0-c17f49581d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:37.629468 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd6826-f06f-45ea-83e0-c17f49581d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:37.629748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd6826-f06f-45ea-83e0-c17f49581d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:37.630020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd6826-f06f-45ea-83e0-c17f49581d4e admin admin] PATH_INFO: `/v3/users/9e6403d8c2f5412099d011dc8764a99c` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:37.631188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd6826-f06f-45ea-83e0-c17f49581d4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e6403d8c2f5412099d011dc8764a99c)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:37.633142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd6826-f06f-45ea-83e0-c17f49581d4e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:37.678919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-72fd6826-f06f-45ea-83e0-c17f49581d4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e6403d8c2f5412099d011dc8764a99c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:37.686168 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-72fd6826-f06f-45ea-83e0-c17f49581d4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e6403d8c2f5412099d011dc8764a99c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:37.686800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-72fd6826-f06f-45ea-83e0-c17f49581d4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e6403d8c2f5412099d011dc8764a99c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:37.698748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-72fd6826-f06f-45ea-83e0-c17f49581d4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e6403d8c2f5412099d011dc8764a99c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:37.699371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-72fd6826-f06f-45ea-83e0-c17f49581d4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e6403d8c2f5412099d011dc8764a99c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:37.711201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1138/2260] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:37 2022] DELETE /identity/v3/users/9e6403d8c2f5412099d011dc8764a99c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:37.986400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:37.988496 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:38.003546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049b462-9dfb-4e70-8376-5c9ec2add896 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:38.066220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:38.067097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.067415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.067696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb admin admin] PATH_INFO: `/v3/projects/aae329fd3e004af7aeb2e4bf77845c74` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.068964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae329fd3e004af7aeb2e4bf77845c74)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:38.070887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:38.117419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae329fd3e004af7aeb2e4bf77845c74', 'request_id': 'req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:38.125994 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae329fd3e004af7aeb2e4bf77845c74', 'request_id': 'req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:38.132317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae329fd3e004af7aeb2e4bf77845c74', 'request_id': 'req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:38.142836 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb admin admin] The token cache is being invalidate because project aae329fd3e004af7aeb2e4bf77845c74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:38.143124 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:38.143752 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-7fbe0720-ca5d-4539-a5f6-404562f6a6eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:38.145291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1123/2261] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:37 2022] DELETE /identity/v3/projects/aae329fd3e004af7aeb2e4bf77845c74 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:38.150112 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcada66-d212-4354-b1c1-b89db963cd24 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.150661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcada66-d212-4354-b1c1-b89db963cd24 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.150750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcada66-d212-4354-b1c1-b89db963cd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.168727 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-dfcada66-d212-4354-b1c1-b89db963cd24 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:38.169922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-dfcada66-d212-4354-b1c1-b89db963cd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:38.188595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:38.229874 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1139/2262] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:38.238633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fe2f7-bb8d-45b9-bcb6-91f1082c8220 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.238923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fe2f7-bb8d-45b9-bcb6-91f1082c8220 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.239230 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fe2f7-bb8d-45b9-bcb6-91f1082c8220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.259120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-129fe2f7-bb8d-45b9-bcb6-91f1082c8220 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:38.260997 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-129fe2f7-bb8d-45b9-bcb6-91f1082c8220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:38.268954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1124/2263] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:38.273092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcada66-d212-4354-b1c1-b89db963cd24 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:38.290126 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcada66-d212-4354-b1c1-b89db963cd24 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:38.294867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e3360-6dd2-4b67-a5a4-e52c69764a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:38.295572 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e3360-6dd2-4b67-a5a4-e52c69764a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.295808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e3360-6dd2-4b67-a5a4-e52c69764a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.296145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e3360-6dd2-4b67-a5a4-e52c69764a0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.296923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e3360-6dd2-4b67-a5a4-e52c69764a0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:21:38.297218 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e3360-6dd2-4b67-a5a4-e52c69764a0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:38.298814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e3360-6dd2-4b67-a5a4-e52c69764a0b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:38.310867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1140/2264] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:21:38.315165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fe2f7-bb8d-45b9-bcb6-91f1082c8220 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:38.325086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fe2f7-bb8d-45b9-bcb6-91f1082c8220 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:38.325907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:38.331645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ae007-5ed6-4970-9abb-7c6297c3a605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:38.332598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ae007-5ed6-4970-9abb-7c6297c3a605 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.333134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ae007-5ed6-4970-9abb-7c6297c3a605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.333605 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ae007-5ed6-4970-9abb-7c6297c3a605 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.334665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ae007-5ed6-4970-9abb-7c6297c3a605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:38.336886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ae007-5ed6-4970-9abb-7c6297c3a605 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:38.362880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1125/2265] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:38 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:38.366975 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e3360-6dd2-4b67-a5a4-e52c69764a0b admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:38.378803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e3360-6dd2-4b67-a5a4-e52c69764a0b admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:38.440674 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65a218-8f38-4805-b871-63622cb33a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:38.441379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65a218-8f38-4805-b871-63622cb33a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.441703 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65a218-8f38-4805-b871-63622cb33a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.441995 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65a218-8f38-4805-b871-63622cb33a81 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.442702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65a218-8f38-4805-b871-63622cb33a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:38.444667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65a218-8f38-4805-b871-63622cb33a81 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:38.464251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1141/2266] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:38 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:38.468954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ae007-5ed6-4970-9abb-7c6297c3a605 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:38.479499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ae007-5ed6-4970-9abb-7c6297c3a605 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:38.485119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d89be3-48df-4afa-8eef-9ca53a66cef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:38.485784 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d89be3-48df-4afa-8eef-9ca53a66cef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.486072 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d89be3-48df-4afa-8eef-9ca53a66cef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.486358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d89be3-48df-4afa-8eef-9ca53a66cef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.487088 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d89be3-48df-4afa-8eef-9ca53a66cef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:38.488721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d89be3-48df-4afa-8eef-9ca53a66cef2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:38.499961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1126/2267] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:21:38.504294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65a218-8f38-4805-b871-63622cb33a81 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:38.515403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65a218-8f38-4805-b871-63622cb33a81 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:38.520664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961f943-d59b-4314-acf0-50cc2d5a4915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:38.521341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961f943-d59b-4314-acf0-50cc2d5a4915 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.521905 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961f943-d59b-4314-acf0-50cc2d5a4915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.521905 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961f943-d59b-4314-acf0-50cc2d5a4915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.522744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961f943-d59b-4314-acf0-50cc2d5a4915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:38.524396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961f943-d59b-4314-acf0-50cc2d5a4915 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:38.536832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1142/2268] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:21:38.541606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d89be3-48df-4afa-8eef-9ca53a66cef2 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:38.553194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d89be3-48df-4afa-8eef-9ca53a66cef2 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:38.559077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81bab0-d7ef-4dd4-9145-e2d6560fb4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:38.560001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81bab0-d7ef-4dd4-9145-e2d6560fb4bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.560330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81bab0-d7ef-4dd4-9145-e2d6560fb4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.560615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81bab0-d7ef-4dd4-9145-e2d6560fb4bf admin admin] PATH_INFO: `/v3/projects/8a3b5f27de44444d88c954c095e6b03d/users/964ec188d8cc486db73c36049affbeda/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.577898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81bab0-d7ef-4dd4-9145-e2d6560fb4bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a3b5f27de44444d88c954c095e6b03d, user_id=964ec188d8cc486db73c36049affbeda, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:38.580722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81bab0-d7ef-4dd4-9145-e2d6560fb4bf admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:38.598994 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1127/2269] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:38 2022] PUT /identity/v3/projects/8a3b5f27de44444d88c954c095e6b03d/users/964ec188d8cc486db73c36049affbeda/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:38.611043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd14a4d-1706-415b-b464-d57918ab8ada None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.611140 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd14a4d-1706-415b-b464-d57918ab8ada None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.611424 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd14a4d-1706-415b-b464-d57918ab8ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.665266 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-cdd14a4d-1706-415b-b464-d57918ab8ada None None] MFA Rules not processed for user `964ec188d8cc486db73c36049affbeda`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:38.666717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cdd14a4d-1706-415b-b464-d57918ab8ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:38.740002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1143/2270] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:38.745124 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81bab0-d7ef-4dd4-9145-e2d6560fb4bf admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:38.809882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81bab0-d7ef-4dd4-9145-e2d6560fb4bf admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:38.814502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a28e3-06ea-405e-aab6-cc83e22fe833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:38.815163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a28e3-06ea-405e-aab6-cc83e22fe833 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.815451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a28e3-06ea-405e-aab6-cc83e22fe833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.815729 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a28e3-06ea-405e-aab6-cc83e22fe833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.816535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a28e3-06ea-405e-aab6-cc83e22fe833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:38.818445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a28e3-06ea-405e-aab6-cc83e22fe833 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:38.843823 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1128/2271] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:38 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:38.847781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd14a4d-1706-415b-b464-d57918ab8ada None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:38.858923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd14a4d-1706-415b-b464-d57918ab8ada None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:38.875781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:38.914936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee716c-08ce-4dfd-8e9e-e2580031c690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:38.915605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee716c-08ce-4dfd-8e9e-e2580031c690 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.915893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee716c-08ce-4dfd-8e9e-e2580031c690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.916791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee716c-08ce-4dfd-8e9e-e2580031c690 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.917473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee716c-08ce-4dfd-8e9e-e2580031c690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:38.919408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee716c-08ce-4dfd-8e9e-e2580031c690 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:38.939085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1144/2272] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:38 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:38.943178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a28e3-06ea-405e-aab6-cc83e22fe833 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:38.953379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a28e3-06ea-405e-aab6-cc83e22fe833 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:38.958010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692029b2-751d-48ea-9ca8-3db83c4f5613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:38.958667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692029b2-751d-48ea-9ca8-3db83c4f5613 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.958956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692029b2-751d-48ea-9ca8-3db83c4f5613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.959238 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692029b2-751d-48ea-9ca8-3db83c4f5613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:38.959960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692029b2-751d-48ea-9ca8-3db83c4f5613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:38.961595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692029b2-751d-48ea-9ca8-3db83c4f5613 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:38.971902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1129/2273] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:38.975845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee716c-08ce-4dfd-8e9e-e2580031c690 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:38.986866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee716c-08ce-4dfd-8e9e-e2580031c690 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:38.992609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdb489-cc57-489a-ab6c-84b5f5fb1597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:38.993420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdb489-cc57-489a-ab6c-84b5f5fb1597 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:38.993702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdb489-cc57-489a-ab6c-84b5f5fb1597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:38.994044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdb489-cc57-489a-ab6c-84b5f5fb1597 admin admin] PATH_INFO: `/v3/projects/c0611d21abae4be1928d738677503edc/users/d190c0cc6675425ab89f2b16c4891d33/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:39.010724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdb489-cc57-489a-ab6c-84b5f5fb1597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0611d21abae4be1928d738677503edc, user_id=d190c0cc6675425ab89f2b16c4891d33, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:39.013422 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdb489-cc57-489a-ab6c-84b5f5fb1597 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:39.027098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1145/2274] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:38 2022] PUT /identity/v3/projects/c0611d21abae4be1928d738677503edc/users/d190c0cc6675425ab89f2b16c4891d33/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:39.031319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692029b2-751d-48ea-9ca8-3db83c4f5613 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:39.042196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692029b2-751d-48ea-9ca8-3db83c4f5613 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:39.100354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a354037-e991-4619-81e4-763dc731b1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:39.101038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a354037-e991-4619-81e4-763dc731b1af admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:39.101456 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a354037-e991-4619-81e4-763dc731b1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:39.101811 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a354037-e991-4619-81e4-763dc731b1af admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:39.102903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a354037-e991-4619-81e4-763dc731b1af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:39.106405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a354037-e991-4619-81e4-763dc731b1af admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:39.122035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1130/2275] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:21:39.123493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdb489-cc57-489a-ab6c-84b5f5fb1597 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:39.135580 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdb489-cc57-489a-ab6c-84b5f5fb1597 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:39.152147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7162f8-7ea0-4c81-b1d6-c6c33aad2a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:39.152147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7162f8-7ea0-4c81-b1d6-c6c33aad2a7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:39.152147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7162f8-7ea0-4c81-b1d6-c6c33aad2a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:39.152147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7162f8-7ea0-4c81-b1d6-c6c33aad2a7f admin admin] PATH_INFO: `/v3/projects/c0611d21abae4be1928d738677503edc/users/d190c0cc6675425ab89f2b16c4891d33/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:39.152989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7162f8-7ea0-4c81-b1d6-c6c33aad2a7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0611d21abae4be1928d738677503edc, user_id=d190c0cc6675425ab89f2b16c4891d33, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:39.152989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7162f8-7ea0-4c81-b1d6-c6c33aad2a7f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:39.162777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1146/2276] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:21:39 2022] PUT /identity/v3/projects/c0611d21abae4be1928d738677503edc/users/d190c0cc6675425ab89f2b16c4891d33/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:39.174005 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f8275-e8ec-40cd-acd4-7e76413a7a6c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:39.174297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f8275-e8ec-40cd-acd4-7e76413a7a6c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:39.174600 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f8275-e8ec-40cd-acd4-7e76413a7a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:39.200093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-fb4f8275-e8ec-40cd-acd4-7e76413a7a6c None None] MFA Rules not processed for user `d190c0cc6675425ab89f2b16c4891d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:39.201275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-fb4f8275-e8ec-40cd-acd4-7e76413a7a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:39.206106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:39.271412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1131/2277] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:39.282402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7162f8-7ea0-4c81-b1d6-c6c33aad2a7f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:39.284131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-4e7162f8-7ea0-4c81-b1d6-c6c33aad2a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:39.340574 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7162f8-7ea0-4c81-b1d6-c6c33aad2a7f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:39.344173 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fea57-1629-4ac6-97a8-cd6b230fcb82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:39.344804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fea57-1629-4ac6-97a8-cd6b230fcb82 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:39.345087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fea57-1629-4ac6-97a8-cd6b230fcb82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:39.345364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fea57-1629-4ac6-97a8-cd6b230fcb82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:39.402438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745fea57-1629-4ac6-97a8-cd6b230fcb82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:39.403878 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745fea57-1629-4ac6-97a8-cd6b230fcb82 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:39.412983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1147/2278] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:45.826678 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f8275-e8ec-40cd-acd4-7e76413a7a6c None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:45.828216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-fb4f8275-e8ec-40cd-acd4-7e76413a7a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:45.851271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f8275-e8ec-40cd-acd4-7e76413a7a6c None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:45.855388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4b600-3e0d-440d-9bc5-95cf05aa81e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:45.856094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4b600-3e0d-440d-9bc5-95cf05aa81e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:45.856552 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4b600-3e0d-440d-9bc5-95cf05aa81e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:45.857022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4b600-3e0d-440d-9bc5-95cf05aa81e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:45.868672 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4b600-3e0d-440d-9bc5-95cf05aa81e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:45.872871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4b600-3e0d-440d-9bc5-95cf05aa81e4 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:45.886073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1132/2279] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:21:47.483113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fe5ec-23d6-4371-bcef-e44e62e75ef2 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:47.483627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fe5ec-23d6-4371-bcef-e44e62e75ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:47.483724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fe5ec-23d6-4371-bcef-e44e62e75ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:47.500988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-431fe5ec-23d6-4371-bcef-e44e62e75ef2 None None] MFA Rules not processed for user `c11f44eb67cd4923a840171833bb572a`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:47.502188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-431fe5ec-23d6-4371-bcef-e44e62e75ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:47.522788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:47.557676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1148/2280] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:54.856410 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd26f3-3654-4e2f-a08f-f71c4914a271 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:54.856895 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd26f3-3654-4e2f-a08f-f71c4914a271 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:54.856991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd26f3-3654-4e2f-a08f-f71c4914a271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:54.874974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-50dd26f3-3654-4e2f-a08f-f71c4914a271 None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:54.876203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-50dd26f3-3654-4e2f-a08f-f71c4914a271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:54.930845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1133/2281] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:21:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:54.971593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fe5ec-23d6-4371-bcef-e44e62e75ef2 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:54.973134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-431fe5ec-23d6-4371-bcef-e44e62e75ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:55.042472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fe5ec-23d6-4371-bcef-e44e62e75ef2 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:55.047323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18158f-05da-4fb2-8607-70e63cdba3e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:55.047970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18158f-05da-4fb2-8607-70e63cdba3e0 service swift] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:55.048284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18158f-05da-4fb2-8607-70e63cdba3e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:55.048566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18158f-05da-4fb2-8607-70e63cdba3e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:55.057189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18158f-05da-4fb2-8607-70e63cdba3e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:55.059502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18158f-05da-4fb2-8607-70e63cdba3e0 service swift] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:55.067720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1149/2282] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:21:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:21:56.362467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd26f3-3654-4e2f-a08f-f71c4914a271 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:56.514635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd26f3-3654-4e2f-a08f-f71c4914a271 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:56.528526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:56.529573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:56.531724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:56.532014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] PATH_INFO: `/v3/users/964ec188d8cc486db73c36049affbeda` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:56.549683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=964ec188d8cc486db73c36049affbeda)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:56.551698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:56.624606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:56.646959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '964ec188d8cc486db73c36049affbeda'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:56.647865 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '964ec188d8cc486db73c36049affbeda'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:56.662171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '964ec188d8cc486db73c36049affbeda'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:56.686436 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '964ec188d8cc486db73c36049affbeda'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:56.687521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '964ec188d8cc486db73c36049affbeda'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:56.708597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1134/2283] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:56 2022] DELETE /identity/v3/users/964ec188d8cc486db73c36049affbeda => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:56.715074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619105b-5944-4efc-9a2a-cefb9d345b7d None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:56.715886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619105b-5944-4efc-9a2a-cefb9d345b7d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:56.716327 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619105b-5944-4efc-9a2a-cefb9d345b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:56.739110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6619105b-5944-4efc-9a2a-cefb9d345b7d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:56.740842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6619105b-5944-4efc-9a2a-cefb9d345b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:56.849717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1150/2284] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:21:56.855768 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:56.857732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:56.945563 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816ceda-1f82-43c5-8134-a36ecbeb7ea5 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:56.950849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdc849-8bdf-409a-93d8-30c8d0b015aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:56.953179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdc849-8bdf-409a-93d8-30c8d0b015aa service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:56.953585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdc849-8bdf-409a-93d8-30c8d0b015aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:56.953983 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdc849-8bdf-409a-93d8-30c8d0b015aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:56.972017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdc849-8bdf-409a-93d8-30c8d0b015aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:56.975258 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdc849-8bdf-409a-93d8-30c8d0b015aa service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:57.000892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1135/2285] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:21:57.434804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619105b-5944-4efc-9a2a-cefb9d345b7d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:57.436512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6619105b-5944-4efc-9a2a-cefb9d345b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:57.477054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619105b-5944-4efc-9a2a-cefb9d345b7d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:57.487320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdc849-8bdf-409a-93d8-30c8d0b015aa service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:57.487857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:57.488992 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:57.489606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:57.490006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] PATH_INFO: `/v3/projects/8a3b5f27de44444d88c954c095e6b03d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:57.490133 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c8cdc849-8bdf-409a-93d8-30c8d0b015aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:57.492394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a3b5f27de44444d88c954c095e6b03d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:57.495174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:57.512210 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdc849-8bdf-409a-93d8-30c8d0b015aa service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:57.518014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b6232-6aea-4136-96f6-99d381b06bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:57.518907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b6232-6aea-4136-96f6-99d381b06bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:57.519196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b6232-6aea-4136-96f6-99d381b06bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:57.519489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b6232-6aea-4136-96f6-99d381b06bc2 admin admin] PATH_INFO: `/v3/users/31ad277d407e4d07afa20304a4bf2c5f` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:57.521195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b6232-6aea-4136-96f6-99d381b06bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31ad277d407e4d07afa20304a4bf2c5f)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:57.524312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b6232-6aea-4136-96f6-99d381b06bc2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:57.568815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a3b5f27de44444d88c954c095e6b03d', 'request_id': 'req-7ebf7e5e-5dff-4021-a771-bf949cf153c9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:57.578070 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:57.580576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a3b5f27de44444d88c954c095e6b03d', 'request_id': 'req-7ebf7e5e-5dff-4021-a771-bf949cf153c9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:57.592941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a50b6232-6aea-4136-96f6-99d381b06bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31ad277d407e4d07afa20304a4bf2c5f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:57.593767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a50b6232-6aea-4136-96f6-99d381b06bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31ad277d407e4d07afa20304a4bf2c5f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:57.604675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a50b6232-6aea-4136-96f6-99d381b06bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31ad277d407e4d07afa20304a4bf2c5f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:57.608849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a3b5f27de44444d88c954c095e6b03d', 'request_id': 'req-7ebf7e5e-5dff-4021-a771-bf949cf153c9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:57.628831 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a50b6232-6aea-4136-96f6-99d381b06bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31ad277d407e4d07afa20304a4bf2c5f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:57.629763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a50b6232-6aea-4136-96f6-99d381b06bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31ad277d407e4d07afa20304a4bf2c5f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:57.635071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] The token cache is being invalidate because project 8a3b5f27de44444d88c954c095e6b03d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:57.635546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:57.636755 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:57.639451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1151/2286] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:57 2022] DELETE /identity/v3/projects/8a3b5f27de44444d88c954c095e6b03d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:57.643987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:57.645837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:57.654170 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1136/2287] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:57 2022] DELETE /identity/v3/users/31ad277d407e4d07afa20304a4bf2c5f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:57.660801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bc5b0-81cf-4e7e-8121-c5c706dd500a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:57.661097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bc5b0-81cf-4e7e-8121-c5c706dd500a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:57.661376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bc5b0-81cf-4e7e-8121-c5c706dd500a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:57.689583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-715bc5b0-81cf-4e7e-8121-c5c706dd500a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:57.691164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-715bc5b0-81cf-4e7e-8121-c5c706dd500a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:57.724594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:57.829135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf7e5e-5dff-4021-a771-bf949cf153c9 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:57.830664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1137/2288] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:57.842137 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bc5b0-81cf-4e7e-8121-c5c706dd500a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:57.843334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:57.844253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-715bc5b0-81cf-4e7e-8121-c5c706dd500a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:57.846229 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:57.865807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:57.866222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] PATH_INFO: `/v3/users/d190c0cc6675425ab89f2b16c4891d33` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:57.872153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d190c0cc6675425ab89f2b16c4891d33)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:57.877335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:57.982550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd190c0cc6675425ab89f2b16c4891d33'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:57.997956 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd190c0cc6675425ab89f2b16c4891d33'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:57.998799 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd190c0cc6675425ab89f2b16c4891d33'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:58.006380 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bc5b0-81cf-4e7e-8121-c5c706dd500a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:58.016129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1552c-cba5-42c6-b256-f4e709782f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:58.017270 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1552c-cba5-42c6-b256-f4e709782f2c service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:58.017560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1552c-cba5-42c6-b256-f4e709782f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:58.018843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1552c-cba5-42c6-b256-f4e709782f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:58.041415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd190c0cc6675425ab89f2b16c4891d33'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:58.044161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd190c0cc6675425ab89f2b16c4891d33'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:58.054301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1552c-cba5-42c6-b256-f4e709782f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:58.064175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1552c-cba5-42c6-b256-f4e709782f2c service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:58.077570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1152/2289] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:21:57 2022] DELETE /identity/v3/users/d190c0cc6675425ab89f2b16c4891d33 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:58.088981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1138/2290] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:21:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:21:58.845076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:58.846588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:59.001036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139f6c8-e943-48c0-8a9b-b5fc3ce28eec admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:59.011401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6953d-ccc8-4543-a8e0-b619052cb9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:59.014031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6953d-ccc8-4543-a8e0-b619052cb9b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:59.014446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6953d-ccc8-4543-a8e0-b619052cb9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:59.014821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6953d-ccc8-4543-a8e0-b619052cb9b2 admin admin] PATH_INFO: `/v3/projects/59c145a95ed94a538bafaf99e7429607` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:59.016168 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6953d-ccc8-4543-a8e0-b619052cb9b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c145a95ed94a538bafaf99e7429607)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:59.018632 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6953d-ccc8-4543-a8e0-b619052cb9b2 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:59.027244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:59.113044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-58f6953d-ccc8-4543-a8e0-b619052cb9b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c145a95ed94a538bafaf99e7429607', 'request_id': 'req-58f6953d-ccc8-4543-a8e0-b619052cb9b2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:59.122404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-58f6953d-ccc8-4543-a8e0-b619052cb9b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c145a95ed94a538bafaf99e7429607', 'request_id': 'req-58f6953d-ccc8-4543-a8e0-b619052cb9b2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:59.130831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-58f6953d-ccc8-4543-a8e0-b619052cb9b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c145a95ed94a538bafaf99e7429607', 'request_id': 'req-58f6953d-ccc8-4543-a8e0-b619052cb9b2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:59.155553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-58f6953d-ccc8-4543-a8e0-b619052cb9b2 admin admin] The token cache is being invalidate because project 59c145a95ed94a538bafaf99e7429607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:59.155908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-58f6953d-ccc8-4543-a8e0-b619052cb9b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:59.157951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-58f6953d-ccc8-4543-a8e0-b619052cb9b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:59.172179 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1153/2291] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:58 2022] DELETE /identity/v3/projects/59c145a95ed94a538bafaf99e7429607 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:21:59.339783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1552c-cba5-42c6-b256-f4e709782f2c service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:59.354544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1552c-cba5-42c6-b256-f4e709782f2c service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:59.478335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44172682-b4bf-4196-a604-3f1e5a63fab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:59.479509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44172682-b4bf-4196-a604-3f1e5a63fab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:59.480117 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44172682-b4bf-4196-a604-3f1e5a63fab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:59.480557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44172682-b4bf-4196-a604-3f1e5a63fab3 admin admin] PATH_INFO: `/v3/projects/c0611d21abae4be1928d738677503edc` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:59.482337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44172682-b4bf-4196-a604-3f1e5a63fab3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0611d21abae4be1928d738677503edc)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:59.484518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44172682-b4bf-4196-a604-3f1e5a63fab3 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:59.549606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-44172682-b4bf-4196-a604-3f1e5a63fab3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0611d21abae4be1928d738677503edc', 'request_id': 'req-44172682-b4bf-4196-a604-3f1e5a63fab3'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:59.558190 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-44172682-b4bf-4196-a604-3f1e5a63fab3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0611d21abae4be1928d738677503edc', 'request_id': 'req-44172682-b4bf-4196-a604-3f1e5a63fab3'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:59.565475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-44172682-b4bf-4196-a604-3f1e5a63fab3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0611d21abae4be1928d738677503edc', 'request_id': 'req-44172682-b4bf-4196-a604-3f1e5a63fab3'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:59.577757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-44172682-b4bf-4196-a604-3f1e5a63fab3 admin admin] The token cache is being invalidate because project c0611d21abae4be1928d738677503edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:21:59.578216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-44172682-b4bf-4196-a604-3f1e5a63fab3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:59.578980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-44172682-b4bf-4196-a604-3f1e5a63fab3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:21:59.585190 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1139/2292] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:21:59 2022] DELETE /identity/v3/projects/c0611d21abae4be1928d738677503edc => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:21:59.586591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4b15c-c803-4e15-8662-e3580f011536 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:59.586897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4b15c-c803-4e15-8662-e3580f011536 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:59.587210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4b15c-c803-4e15-8662-e3580f011536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:59.607267 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-cba4b15c-c803-4e15-8662-e3580f011536 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:59.608553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cba4b15c-c803-4e15-8662-e3580f011536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:59.681096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1154/2293] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:21:59.689395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9f0ae-a71a-461f-9b0d-01c043bd6201 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:59.689755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9f0ae-a71a-461f-9b0d-01c043bd6201 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:59.690034 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9f0ae-a71a-461f-9b0d-01c043bd6201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:59.697514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:21:59.715160 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-fee9f0ae-a71a-461f-9b0d-01c043bd6201 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:21:59.716407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-fee9f0ae-a71a-461f-9b0d-01c043bd6201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:21:59.729132 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1140/2294] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:21:59.729480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4b15c-c803-4e15-8662-e3580f011536 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:59.739959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4b15c-c803-4e15-8662-e3580f011536 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:59.746392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ca571-778b-4435-b48e-69fb67a09ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:59.747552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ca571-778b-4435-b48e-69fb67a09ead admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:59.747846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ca571-778b-4435-b48e-69fb67a09ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:59.748227 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ca571-778b-4435-b48e-69fb67a09ead admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:59.749000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ca571-778b-4435-b48e-69fb67a09ead admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:21:59.749316 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ca571-778b-4435-b48e-69fb67a09ead admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:59.751192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ca571-778b-4435-b48e-69fb67a09ead admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:59.767864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1155/2295] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:21:59.768791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9f0ae-a71a-461f-9b0d-01c043bd6201 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:59.787194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9f0ae-a71a-461f-9b0d-01c043bd6201 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:59.807955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b209533-480e-4321-8833-03940712d9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:59.810423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b209533-480e-4321-8833-03940712d9d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:59.810886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b209533-480e-4321-8833-03940712d9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:59.811985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b209533-480e-4321-8833-03940712d9d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:59.814696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b209533-480e-4321-8833-03940712d9d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:59.816872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b209533-480e-4321-8833-03940712d9d6 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:21:59.883050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1141/2296] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:21:59 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:21:59.885379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ca571-778b-4435-b48e-69fb67a09ead admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:21:59.900686 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ca571-778b-4435-b48e-69fb67a09ead admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:21:59.986312 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971e959-7fdf-46de-bd6d-bbf7c986b2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:21:59.987071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971e959-7fdf-46de-bd6d-bbf7c986b2ff admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:21:59.987365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971e959-7fdf-46de-bd6d-bbf7c986b2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:21:59.987861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971e959-7fdf-46de-bd6d-bbf7c986b2ff admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:21:59.988766 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971e959-7fdf-46de-bd6d-bbf7c986b2ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:21:59.991723 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971e959-7fdf-46de-bd6d-bbf7c986b2ff admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:00.019986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1156/2297] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:21:59 2022] POST /identity/v3/users => generated 290 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:22:00.023618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b209533-480e-4321-8833-03940712d9d6 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:00.038669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b209533-480e-4321-8833-03940712d9d6 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:00.044029 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6b301-2fd7-4b79-a98f-ca426e189029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:00.044886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6b301-2fd7-4b79-a98f-ca426e189029 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:00.045180 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6b301-2fd7-4b79-a98f-ca426e189029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:00.045511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6b301-2fd7-4b79-a98f-ca426e189029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:00.046260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6b301-2fd7-4b79-a98f-ca426e189029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:00.048236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6b301-2fd7-4b79-a98f-ca426e189029 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:00.063710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1142/2298] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:22:00.070396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971e959-7fdf-46de-bd6d-bbf7c986b2ff admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:00.082237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971e959-7fdf-46de-bd6d-bbf7c986b2ff admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:00.087693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca19ae-272d-480b-840e-64614005c4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:00.088443 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca19ae-272d-480b-840e-64614005c4ae admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:00.088915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca19ae-272d-480b-840e-64614005c4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:00.089355 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca19ae-272d-480b-840e-64614005c4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:00.090149 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca19ae-272d-480b-840e-64614005c4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:00.091883 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca19ae-272d-480b-840e-64614005c4ae admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:00.098222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:00.104928 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1157/2299] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:22:00.109066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6b301-2fd7-4b79-a98f-ca426e189029 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:00.121138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6b301-2fd7-4b79-a98f-ca426e189029 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:00.126175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f88a5-3b77-4f21-9291-468f6378c307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:00.127069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f88a5-3b77-4f21-9291-468f6378c307 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:00.127360 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f88a5-3b77-4f21-9291-468f6378c307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:00.127641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f88a5-3b77-4f21-9291-468f6378c307 admin admin] PATH_INFO: `/v3/projects/71a5393056b74f1e871bb50775a40554/users/7615cf807ce746ada15efa0ff1cb80d7/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:00.144870 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f88a5-3b77-4f21-9291-468f6378c307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a5393056b74f1e871bb50775a40554, user_id=7615cf807ce746ada15efa0ff1cb80d7, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:00.147720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f88a5-3b77-4f21-9291-468f6378c307 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:00.166374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1143/2300] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:22:00 2022] PUT /identity/v3/projects/71a5393056b74f1e871bb50775a40554/users/7615cf807ce746ada15efa0ff1cb80d7/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:22:00.172329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d96d5-f813-4ae9-b0b9-050ab5cc590f None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:00.173073 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d96d5-f813-4ae9-b0b9-050ab5cc590f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:00.173566 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d96d5-f813-4ae9-b0b9-050ab5cc590f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:00.196770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-b11d96d5-f813-4ae9-b0b9-050ab5cc590f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:00.198621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b11d96d5-f813-4ae9-b0b9-050ab5cc590f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:00.285800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1158/2301] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:00.296292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f88a5-3b77-4f21-9291-468f6378c307 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:00.298065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-af9f88a5-3b77-4f21-9291-468f6378c307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:00.365179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f88a5-3b77-4f21-9291-468f6378c307 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:00.369698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed833df1-ed0e-4da5-8960-99f00a976091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:00.371096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed833df1-ed0e-4da5-8960-99f00a976091 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:00.371384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed833df1-ed0e-4da5-8960-99f00a976091 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:00.371662 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed833df1-ed0e-4da5-8960-99f00a976091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:00.381306 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed833df1-ed0e-4da5-8960-99f00a976091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:00.382942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed833df1-ed0e-4da5-8960-99f00a976091 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:00.405543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1144/2302] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:22:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:22:01.411115 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf789a9-78f9-41e8-b5c3-b00b324ec0c3 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:01.412035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf789a9-78f9-41e8-b5c3-b00b324ec0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:01.412642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf789a9-78f9-41e8-b5c3-b00b324ec0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:01.510405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-daf789a9-78f9-41e8-b5c3-b00b324ec0c3 None None] MFA Rules not processed for user `7615cf807ce746ada15efa0ff1cb80d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:01.513373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-daf789a9-78f9-41e8-b5c3-b00b324ec0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:01.616329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1159/2303] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:01.628083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed833df1-ed0e-4da5-8960-99f00a976091 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:01.629487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ed833df1-ed0e-4da5-8960-99f00a976091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:01.691583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed833df1-ed0e-4da5-8960-99f00a976091 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:01.695511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41911daf-fbc1-44fe-b375-d7b4e26b044b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:01.696197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41911daf-fbc1-44fe-b375-d7b4e26b044b service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:01.696485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41911daf-fbc1-44fe-b375-d7b4e26b044b service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:01.696757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41911daf-fbc1-44fe-b375-d7b4e26b044b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:01.706494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41911daf-fbc1-44fe-b375-d7b4e26b044b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:01.707921 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41911daf-fbc1-44fe-b375-d7b4e26b044b service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:01.718741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1145/2304] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:22:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:22:03.206637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe277a-e8cd-4616-8155-23b562fc20bc None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:03.207258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe277a-e8cd-4616-8155-23b562fc20bc None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:03.207258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe277a-e8cd-4616-8155-23b562fc20bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:03.225821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-4ffe277a-e8cd-4616-8155-23b562fc20bc None None] MFA Rules not processed for user `7615cf807ce746ada15efa0ff1cb80d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:03.227212 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-4ffe277a-e8cd-4616-8155-23b562fc20bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:03.243087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1160/2305] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:10.574690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32147e-4e4a-4c88-8b89-aa2e5728222e None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:10.575286 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32147e-4e4a-4c88-8b89-aa2e5728222e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:10.576812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32147e-4e4a-4c88-8b89-aa2e5728222e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:10.611318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-8b32147e-4e4a-4c88-8b89-aa2e5728222e None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:10.614736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-8b32147e-4e4a-4c88-8b89-aa2e5728222e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:10.654760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:10.730075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1146/2306] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:22:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:10.765329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe277a-e8cd-4616-8155-23b562fc20bc None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:10.767729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-4ffe277a-e8cd-4616-8155-23b562fc20bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:10.966928 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:10.996866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe277a-e8cd-4616-8155-23b562fc20bc None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:11.007719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9359f1-7ea3-4078-8cdc-e5ec131deb86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:11.011339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9359f1-7ea3-4078-8cdc-e5ec131deb86 service swift] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:11.012249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9359f1-7ea3-4078-8cdc-e5ec131deb86 service swift] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:11.012773 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9359f1-7ea3-4078-8cdc-e5ec131deb86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:11.026239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9359f1-7ea3-4078-8cdc-e5ec131deb86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:11.029394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9359f1-7ea3-4078-8cdc-e5ec131deb86 service swift] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:11.042064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1161/2307] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:22:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:22:12.235940 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2070310-21e9-4160-b685-e8a7f7eb4aa1 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:12.237603 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2070310-21e9-4160-b685-e8a7f7eb4aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:12.237979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2070310-21e9-4160-b685-e8a7f7eb4aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:12.265695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f2070310-21e9-4160-b685-e8a7f7eb4aa1 None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:12.266938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f2070310-21e9-4160-b685-e8a7f7eb4aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:12.284352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1147/2308] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jun 17 07:22:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:12.298666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9359f1-7ea3-4078-8cdc-e5ec131deb86 service swift] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:12.320823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9359f1-7ea3-4078-8cdc-e5ec131deb86 service swift] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:12.333266 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c72c32-3af5-4f6d-a487-27d1b2e553d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:12.334242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c72c32-3af5-4f6d-a487-27d1b2e553d4 service swift] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:12.334745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c72c32-3af5-4f6d-a487-27d1b2e553d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:12.336164 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c72c32-3af5-4f6d-a487-27d1b2e553d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:12.346503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c72c32-3af5-4f6d-a487-27d1b2e553d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:12.348468 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c72c32-3af5-4f6d-a487-27d1b2e553d4 service swift] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:12.361786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1162/2309] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:22:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:22:12.695366 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713cc7c-9f09-4a9b-ae66-f237d736eab1 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:12.695652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713cc7c-9f09-4a9b-ae66-f237d736eab1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:12.696055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713cc7c-9f09-4a9b-ae66-f237d736eab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:12.723719 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-1713cc7c-9f09-4a9b-ae66-f237d736eab1 None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:12.725359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-1713cc7c-9f09-4a9b-ae66-f237d736eab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:12.741804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1148/2310] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:22:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:12.751612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c72c32-3af5-4f6d-a487-27d1b2e553d4 service swift] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:12.763260 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c72c32-3af5-4f6d-a487-27d1b2e553d4 service swift] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:12.769049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b1699-2118-4aa5-9a12-f17f8865a6a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:12.770443 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b1699-2118-4aa5-9a12-f17f8865a6a9 service swift] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:12.771006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b1699-2118-4aa5-9a12-f17f8865a6a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:12.771446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b1699-2118-4aa5-9a12-f17f8865a6a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:12.781230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b1699-2118-4aa5-9a12-f17f8865a6a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:12.783439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b1699-2118-4aa5-9a12-f17f8865a6a9 service swift] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:12.794189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1163/2311] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:22:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:22:14.926961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153f02a-2eab-4d75-95a5-a1e05c2530d6 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:14.927535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153f02a-2eab-4d75-95a5-a1e05c2530d6 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:14.927648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153f02a-2eab-4d75-95a5-a1e05c2530d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:14.949370 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-6153f02a-2eab-4d75-95a5-a1e05c2530d6 None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:14.951231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6153f02a-2eab-4d75-95a5-a1e05c2530d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:14.959845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1149/2312] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:22:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:14.978454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b1699-2118-4aa5-9a12-f17f8865a6a9 service swift] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:14.996526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b1699-2118-4aa5-9a12-f17f8865a6a9 service swift] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:15.002559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e4128-a876-44e2-8ee1-17beda3b0c5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:15.004023 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e4128-a876-44e2-8ee1-17beda3b0c5c service swift] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:15.004439 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e4128-a876-44e2-8ee1-17beda3b0c5c service swift] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:15.004837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e4128-a876-44e2-8ee1-17beda3b0c5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:15.019322 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e4128-a876-44e2-8ee1-17beda3b0c5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:15.021242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e4128-a876-44e2-8ee1-17beda3b0c5c service swift] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:15.031602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1164/2313] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:22:15.450426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fd17b-fa84-4a3b-b089-e730b4741ad8 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:15.450717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fd17b-fa84-4a3b-b089-e730b4741ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:15.450996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fd17b-fa84-4a3b-b089-e730b4741ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:15.470764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f48fd17b-fa84-4a3b-b089-e730b4741ad8 None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:15.472265 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f48fd17b-fa84-4a3b-b089-e730b4741ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:15.480285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1150/2314] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jun 17 07:22:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:15.505784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e4128-a876-44e2-8ee1-17beda3b0c5c service swift] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:15.521188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e4128-a876-44e2-8ee1-17beda3b0c5c service swift] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:15.528341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b862be-822f-4086-83ed-f7fc7d720121 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:15.529086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b862be-822f-4086-83ed-f7fc7d720121 service swift] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:15.529476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b862be-822f-4086-83ed-f7fc7d720121 service swift] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:15.529822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b862be-822f-4086-83ed-f7fc7d720121 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:15.541591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b862be-822f-4086-83ed-f7fc7d720121 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:15.543522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b862be-822f-4086-83ed-f7fc7d720121 service swift] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:15.560008 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1165/2315] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:22:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:22:16.037650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669f887-c9d3-418f-97b6-8d952efea482 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:16.038159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669f887-c9d3-418f-97b6-8d952efea482 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:16.038744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669f887-c9d3-418f-97b6-8d952efea482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:16.067081 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-7669f887-c9d3-418f-97b6-8d952efea482 None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:16.068453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7669f887-c9d3-418f-97b6-8d952efea482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:16.086970 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1151/2316] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jun 17 07:22:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:16.119674 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b862be-822f-4086-83ed-f7fc7d720121 service swift] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:16.141916 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b862be-822f-4086-83ed-f7fc7d720121 service swift] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:16.147952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5ac48-a421-4d86-84a9-9e35623a897c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:16.148830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5ac48-a421-4d86-84a9-9e35623a897c service swift] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:16.149266 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5ac48-a421-4d86-84a9-9e35623a897c service swift] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:16.149695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5ac48-a421-4d86-84a9-9e35623a897c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:16.169071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5ac48-a421-4d86-84a9-9e35623a897c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:16.170951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5ac48-a421-4d86-84a9-9e35623a897c service swift] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:16.180383 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1166/2317] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:22:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:22:18.201788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669f887-c9d3-418f-97b6-8d952efea482 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:18.203592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7669f887-c9d3-418f-97b6-8d952efea482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:18.272773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669f887-c9d3-418f-97b6-8d952efea482 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:18.281870 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:18.283096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:18.285010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:18.285533 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] PATH_INFO: `/v3/users/dd94a883b0cb4dc4bcc57c221577b29a` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:18.287099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd94a883b0cb4dc4bcc57c221577b29a)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:18.289753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:18.374254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd94a883b0cb4dc4bcc57c221577b29a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:18.375332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd94a883b0cb4dc4bcc57c221577b29a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:18.396782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd94a883b0cb4dc4bcc57c221577b29a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:18.423744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd94a883b0cb4dc4bcc57c221577b29a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:18.424745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd94a883b0cb4dc4bcc57c221577b29a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:18.449484 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1152/2318] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:22:18 2022] DELETE /identity/v3/users/dd94a883b0cb4dc4bcc57c221577b29a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:22:18.454014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1540c-c8dd-47da-bc0d-929453ff2bbe None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:18.454803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1540c-c8dd-47da-bc0d-929453ff2bbe None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:18.455091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1540c-c8dd-47da-bc0d-929453ff2bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:18.503030 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-5aa1540c-c8dd-47da-bc0d-929453ff2bbe None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:18.504342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5aa1540c-c8dd-47da-bc0d-929453ff2bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:18.623166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1167/2319] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:18.635245 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:18.636948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:18.690235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:18.772874 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94411b-f9ff-4037-b0bc-88da609ff0d4 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:18.777394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fc1dc-e460-4e66-b5a6-a560619a4d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:18.778215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fc1dc-e460-4e66-b5a6-a560619a4d6e service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:18.778503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fc1dc-e460-4e66-b5a6-a560619a4d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:18.778781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fc1dc-e460-4e66-b5a6-a560619a4d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:18.799262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4fc1dc-e460-4e66-b5a6-a560619a4d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:18.800759 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4fc1dc-e460-4e66-b5a6-a560619a4d6e service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:18.813890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1153/2320] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:22:19.308474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1540c-c8dd-47da-bc0d-929453ff2bbe None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:19.316990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5aa1540c-c8dd-47da-bc0d-929453ff2bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:19.345858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fc1dc-e460-4e66-b5a6-a560619a4d6e service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:19.347978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9d4fc1dc-e460-4e66-b5a6-a560619a4d6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:19.374300 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1540c-c8dd-47da-bc0d-929453ff2bbe None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:19.394907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729af5df-d5e6-4ca0-bdc2-749130c217c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:19.395818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729af5df-d5e6-4ca0-bdc2-749130c217c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:19.396015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fc1dc-e460-4e66-b5a6-a560619a4d6e service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:19.396225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729af5df-d5e6-4ca0-bdc2-749130c217c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:19.396496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729af5df-d5e6-4ca0-bdc2-749130c217c1 admin admin] PATH_INFO: `/v3/users/9710f6fb31da4df39b04839a29dfdb34` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:19.397911 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729af5df-d5e6-4ca0-bdc2-749130c217c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9710f6fb31da4df39b04839a29dfdb34)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:19.400429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729af5df-d5e6-4ca0-bdc2-749130c217c1 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:19.410609 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f4eaa-d047-41c0-844b-eefada759d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:19.411541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f4eaa-d047-41c0-844b-eefada759d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:19.412015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f4eaa-d047-41c0-844b-eefada759d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:19.412472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f4eaa-d047-41c0-844b-eefada759d09 admin admin] PATH_INFO: `/v3/projects/72db86ca9d3a45b49c7bb140f6b0fa10` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:19.413937 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f4eaa-d047-41c0-844b-eefada759d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72db86ca9d3a45b49c7bb140f6b0fa10)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:19.415904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f4eaa-d047-41c0-844b-eefada759d09 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:19.469920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-729af5df-d5e6-4ca0-bdc2-749130c217c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9710f6fb31da4df39b04839a29dfdb34'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:19.481253 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-729af5df-d5e6-4ca0-bdc2-749130c217c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9710f6fb31da4df39b04839a29dfdb34'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:19.481963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-729af5df-d5e6-4ca0-bdc2-749130c217c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9710f6fb31da4df39b04839a29dfdb34'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:19.492285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:19.502079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-d52f4eaa-d047-41c0-844b-eefada759d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72db86ca9d3a45b49c7bb140f6b0fa10', 'request_id': 'req-d52f4eaa-d047-41c0-844b-eefada759d09'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:19.504984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-729af5df-d5e6-4ca0-bdc2-749130c217c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9710f6fb31da4df39b04839a29dfdb34'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:19.505605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-729af5df-d5e6-4ca0-bdc2-749130c217c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9710f6fb31da4df39b04839a29dfdb34'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:19.527090 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1168/2321] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:22:19 2022] DELETE /identity/v3/users/9710f6fb31da4df39b04839a29dfdb34 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:22:19.541278 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-d52f4eaa-d047-41c0-844b-eefada759d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72db86ca9d3a45b49c7bb140f6b0fa10', 'request_id': 'req-d52f4eaa-d047-41c0-844b-eefada759d09'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:19.549663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-d52f4eaa-d047-41c0-844b-eefada759d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72db86ca9d3a45b49c7bb140f6b0fa10', 'request_id': 'req-d52f4eaa-d047-41c0-844b-eefada759d09'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:19.569364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-d52f4eaa-d047-41c0-844b-eefada759d09 admin admin] The token cache is being invalidate because project 72db86ca9d3a45b49c7bb140f6b0fa10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:22:19.570667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-d52f4eaa-d047-41c0-844b-eefada759d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:19.573421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-d52f4eaa-d047-41c0-844b-eefada759d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:19.577442 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1154/2322] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:22:19 2022] DELETE /identity/v3/projects/72db86ca9d3a45b49c7bb140f6b0fa10 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:22:19.582723 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a17a2-cba9-4598-bce9-94f2822979a4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:19.583095 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a17a2-cba9-4598-bce9-94f2822979a4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:19.583385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a17a2-cba9-4598-bce9-94f2822979a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:19.673131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-f07a17a2-cba9-4598-bce9-94f2822979a4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:19.675268 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f07a17a2-cba9-4598-bce9-94f2822979a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:19.787230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1169/2323] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:19.795892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c8071-0617-4c84-b91d-d5ccc42cc6b8 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:19.796221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c8071-0617-4c84-b91d-d5ccc42cc6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:19.796503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c8071-0617-4c84-b91d-d5ccc42cc6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:19.816899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f20c8071-0617-4c84-b91d-d5ccc42cc6b8 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:19.818369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f20c8071-0617-4c84-b91d-d5ccc42cc6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:19.846790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1155/2324] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:19.859708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a17a2-cba9-4598-bce9-94f2822979a4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:19.886560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a17a2-cba9-4598-bce9-94f2822979a4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:19.916548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee82286-4509-45ed-bd30-59673fbf3e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:19.917562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee82286-4509-45ed-bd30-59673fbf3e45 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:19.918039 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee82286-4509-45ed-bd30-59673fbf3e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:19.918507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee82286-4509-45ed-bd30-59673fbf3e45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:19.919597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee82286-4509-45ed-bd30-59673fbf3e45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:22:19.920159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee82286-4509-45ed-bd30-59673fbf3e45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:19.952284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee82286-4509-45ed-bd30-59673fbf3e45 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:19.991024 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c8071-0617-4c84-b91d-d5ccc42cc6b8 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:19.994493 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f20c8071-0617-4c84-b91d-d5ccc42cc6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:20.035611 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c8071-0617-4c84-b91d-d5ccc42cc6b8 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.053947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d59d29-b06f-41b6-8efa-00d8b8211bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.067972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d59d29-b06f-41b6-8efa-00d8b8211bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.068737 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d59d29-b06f-41b6-8efa-00d8b8211bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.070409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d59d29-b06f-41b6-8efa-00d8b8211bee admin admin] PATH_INFO: `/v3/projects/656ac358f6f44d7180b1122e87dc9c3e` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.076196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d59d29-b06f-41b6-8efa-00d8b8211bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=656ac358f6f44d7180b1122e87dc9c3e)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.083669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d59d29-b06f-41b6-8efa-00d8b8211bee admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:20.101699 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1170/2325] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:22:20.102734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee82286-4509-45ed-bd30-59673fbf3e45 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:20.115310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee82286-4509-45ed-bd30-59673fbf3e45 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.121199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6a132-e0c7-4f97-875e-ab11a5666a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.122219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6a132-e0c7-4f97-875e-ab11a5666a3a admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.122608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6a132-e0c7-4f97-875e-ab11a5666a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.122963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6a132-e0c7-4f97-875e-ab11a5666a3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.123737 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6a132-e0c7-4f97-875e-ab11a5666a3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.125789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6a132-e0c7-4f97-875e-ab11a5666a3a admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:20.158415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1171/2326] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:22:20 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:22:20.162876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6a132-e0c7-4f97-875e-ab11a5666a3a admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:20.171512 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a1d59d29-b06f-41b6-8efa-00d8b8211bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '656ac358f6f44d7180b1122e87dc9c3e', 'request_id': 'req-a1d59d29-b06f-41b6-8efa-00d8b8211bee'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:20.186170 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a1d59d29-b06f-41b6-8efa-00d8b8211bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '656ac358f6f44d7180b1122e87dc9c3e', 'request_id': 'req-a1d59d29-b06f-41b6-8efa-00d8b8211bee'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:20.195058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a1d59d29-b06f-41b6-8efa-00d8b8211bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '656ac358f6f44d7180b1122e87dc9c3e', 'request_id': 'req-a1d59d29-b06f-41b6-8efa-00d8b8211bee'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:20.211534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a1d59d29-b06f-41b6-8efa-00d8b8211bee admin admin] The token cache is being invalidate because project 656ac358f6f44d7180b1122e87dc9c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:22:20.212011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a1d59d29-b06f-41b6-8efa-00d8b8211bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:20.212941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a1d59d29-b06f-41b6-8efa-00d8b8211bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:22:20.216915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1156/2327] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:22:19 2022] DELETE /identity/v3/projects/656ac358f6f44d7180b1122e87dc9c3e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:22:20.224388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a62b0-f015-4e1b-9418-112a791efaae None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.225037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a62b0-f015-4e1b-9418-112a791efaae None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.225890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a62b0-f015-4e1b-9418-112a791efaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.266748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:20.269447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6a132-e0c7-4f97-875e-ab11a5666a3a admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.275964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-333a62b0-f015-4e1b-9418-112a791efaae None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:20.278354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-333a62b0-f015-4e1b-9418-112a791efaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:20.286273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee2a0f-83fe-4bc6-ab9e-2f78def859a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.287300 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee2a0f-83fe-4bc6-ab9e-2f78def859a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.287964 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee2a0f-83fe-4bc6-ab9e-2f78def859a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.288741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee2a0f-83fe-4bc6-ab9e-2f78def859a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.289623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee2a0f-83fe-4bc6-ab9e-2f78def859a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.292364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee2a0f-83fe-4bc6-ab9e-2f78def859a4 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:20.296217 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1157/2328] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:20.304145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a827a-eb74-41d0-b96f-bb19946d1456 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.304700 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a827a-eb74-41d0-b96f-bb19946d1456 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.305272 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a827a-eb74-41d0-b96f-bb19946d1456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.335049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1172/2329] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:22:20 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:22:20.339612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee2a0f-83fe-4bc6-ab9e-2f78def859a4 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:20.342592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-158a827a-eb74-41d0-b96f-bb19946d1456 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:20.345348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f8ee2a0f-83fe-4bc6-ab9e-2f78def859a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:20.345517 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-158a827a-eb74-41d0-b96f-bb19946d1456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:20.374023 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1158/2330] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:20.379053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a827a-eb74-41d0-b96f-bb19946d1456 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:20.381586 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee2a0f-83fe-4bc6-ab9e-2f78def859a4 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.395292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a827a-eb74-41d0-b96f-bb19946d1456 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.403217 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1cdd8-5f99-4bb4-b617-4184b49f95b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.403990 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1cdd8-5f99-4bb4-b617-4184b49f95b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.404426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1cdd8-5f99-4bb4-b617-4184b49f95b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.404772 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1cdd8-5f99-4bb4-b617-4184b49f95b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.405608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1cdd8-5f99-4bb4-b617-4184b49f95b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:22:20.406106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1cdd8-5f99-4bb4-b617-4184b49f95b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.408052 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1cdd8-5f99-4bb4-b617-4184b49f95b2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:20.410825 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdde9b5-ea97-473f-8c56-ef83b92ff6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.416840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdde9b5-ea97-473f-8c56-ef83b92ff6dc admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.417213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdde9b5-ea97-473f-8c56-ef83b92ff6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.417578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdde9b5-ea97-473f-8c56-ef83b92ff6dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.420249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdde9b5-ea97-473f-8c56-ef83b92ff6dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.421980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdde9b5-ea97-473f-8c56-ef83b92ff6dc admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:20.439965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1173/2331] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:22:20.441148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdde9b5-ea97-473f-8c56-ef83b92ff6dc admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:20.442191 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1159/2332] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:22:20.455708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1cdd8-5f99-4bb4-b617-4184b49f95b2 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:20.461507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdde9b5-ea97-473f-8c56-ef83b92ff6dc admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.472641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92958f6b-5c12-4036-b9ec-9ae9887db7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.473385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92958f6b-5c12-4036-b9ec-9ae9887db7ba admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.473891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92958f6b-5c12-4036-b9ec-9ae9887db7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.474188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92958f6b-5c12-4036-b9ec-9ae9887db7ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.474929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92958f6b-5c12-4036-b9ec-9ae9887db7ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.476751 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92958f6b-5c12-4036-b9ec-9ae9887db7ba admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:20.479684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1cdd8-5f99-4bb4-b617-4184b49f95b2 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.479980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:20.488645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc812a-fdf6-425b-ad15-9bfef92f031e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.490424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc812a-fdf6-425b-ad15-9bfef92f031e admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.490870 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc812a-fdf6-425b-ad15-9bfef92f031e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.491300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc812a-fdf6-425b-ad15-9bfef92f031e admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.492396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc812a-fdf6-425b-ad15-9bfef92f031e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.495435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc812a-fdf6-425b-ad15-9bfef92f031e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:20.502925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1174/2333] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:22:20.504198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92958f6b-5c12-4036-b9ec-9ae9887db7ba admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:20.516880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92958f6b-5c12-4036-b9ec-9ae9887db7ba admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.523299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e12ce7-6a34-41ef-b34c-cd0d8d6dc039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.524248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e12ce7-6a34-41ef-b34c-cd0d8d6dc039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.525445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e12ce7-6a34-41ef-b34c-cd0d8d6dc039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.525655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e12ce7-6a34-41ef-b34c-cd0d8d6dc039 admin admin] PATH_INFO: `/v3/projects/52a029f97a9e4142802a9786ae2f6d58/users/a21116ea64234269bba8b0369fb19b59/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.544057 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e12ce7-6a34-41ef-b34c-cd0d8d6dc039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a029f97a9e4142802a9786ae2f6d58, user_id=a21116ea64234269bba8b0369fb19b59, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.547197 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e12ce7-6a34-41ef-b34c-cd0d8d6dc039 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:20.567704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1175/2334] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:22:20 2022] PUT /identity/v3/projects/52a029f97a9e4142802a9786ae2f6d58/users/a21116ea64234269bba8b0369fb19b59/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:22:20.573817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c0b4b-9aa5-43bb-a156-c8dcc37d5ae0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.574110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c0b4b-9aa5-43bb-a156-c8dcc37d5ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.574391 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c0b4b-9aa5-43bb-a156-c8dcc37d5ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.585562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1160/2335] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:22:20 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:22:20.589001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc812a-fdf6-425b-ad15-9bfef92f031e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:20.624337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-010c0b4b-9aa5-43bb-a156-c8dcc37d5ae0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:20.625815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-010c0b4b-9aa5-43bb-a156-c8dcc37d5ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:20.719903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc812a-fdf6-425b-ad15-9bfef92f031e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.750536 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4b43c-4b00-4f02-bbc2-12c3560ff460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.751445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4b43c-4b00-4f02-bbc2-12c3560ff460 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.751839 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4b43c-4b00-4f02-bbc2-12c3560ff460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.752252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4b43c-4b00-4f02-bbc2-12c3560ff460 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.753067 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4b43c-4b00-4f02-bbc2-12c3560ff460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.755277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4b43c-4b00-4f02-bbc2-12c3560ff460 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:20.772796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1176/2336] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:20.799054 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c0b4b-9aa5-43bb-a156-c8dcc37d5ae0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:20.800883 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-010c0b4b-9aa5-43bb-a156-c8dcc37d5ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:20.812035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1161/2337] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:22:20 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:22:20.814739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4b43c-4b00-4f02-bbc2-12c3560ff460 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:20.841327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4b43c-4b00-4f02-bbc2-12c3560ff460 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.858812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62d18a-90fe-4ad7-9c03-5714ffd6058e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.860000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62d18a-90fe-4ad7-9c03-5714ffd6058e admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.860000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62d18a-90fe-4ad7-9c03-5714ffd6058e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.860000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62d18a-90fe-4ad7-9c03-5714ffd6058e admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.860588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62d18a-90fe-4ad7-9c03-5714ffd6058e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.864229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62d18a-90fe-4ad7-9c03-5714ffd6058e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:20.886086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1162/2338] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:22:20.891829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62d18a-90fe-4ad7-9c03-5714ffd6058e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:20.905948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62d18a-90fe-4ad7-9c03-5714ffd6058e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.911637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b3f2f-c98b-43c0-b3d6-481c79fa7dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.912331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b3f2f-c98b-43c0-b3d6-481c79fa7dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.912770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b3f2f-c98b-43c0-b3d6-481c79fa7dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.913061 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b3f2f-c98b-43c0-b3d6-481c79fa7dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.914115 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b3f2f-c98b-43c0-b3d6-481c79fa7dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.915761 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b3f2f-c98b-43c0-b3d6-481c79fa7dd6 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:20.922689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c0b4b-9aa5-43bb-a156-c8dcc37d5ae0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.932221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1163/2339] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:22:20.935961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b3f2f-c98b-43c0-b3d6-481c79fa7dd6 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:20.940769 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6aa70-402e-4fce-b48c-bf0fbe1337c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.941442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6aa70-402e-4fce-b48c-bf0fbe1337c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.941722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6aa70-402e-4fce-b48c-bf0fbe1337c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.941996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6aa70-402e-4fce-b48c-bf0fbe1337c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.956212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b3f2f-c98b-43c0-b3d6-481c79fa7dd6 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:20.957276 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6aa70-402e-4fce-b48c-bf0fbe1337c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.959658 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6aa70-402e-4fce-b48c-bf0fbe1337c5 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:20.961928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f9d90-ff4d-4a07-9cd8-ca077870707e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:20.962375 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f9d90-ff4d-4a07-9cd8-ca077870707e admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:20.962663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f9d90-ff4d-4a07-9cd8-ca077870707e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:20.962940 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f9d90-ff4d-4a07-9cd8-ca077870707e admin admin] PATH_INFO: `/v3/projects/f92a857cef0a43d4907fbc667e30df6d/users/86f2cda914b74f979746f4fb370eaae6/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:20.980701 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f9d90-ff4d-4a07-9cd8-ca077870707e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92a857cef0a43d4907fbc667e30df6d, user_id=86f2cda914b74f979746f4fb370eaae6, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:20.981232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1177/2340] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:22:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:22:20.984850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f9d90-ff4d-4a07-9cd8-ca077870707e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:21.006374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1164/2341] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:22:20 2022] PUT /identity/v3/projects/f92a857cef0a43d4907fbc667e30df6d/users/86f2cda914b74f979746f4fb370eaae6/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:22:21.008453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14487201-3763-4773-a96c-34be5fbc7893 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:21.008991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14487201-3763-4773-a96c-34be5fbc7893 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:21.010058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14487201-3763-4773-a96c-34be5fbc7893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:21.037952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-14487201-3763-4773-a96c-34be5fbc7893 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:21.041969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-14487201-3763-4773-a96c-34be5fbc7893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:21.129531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1178/2342] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:21.138815 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f9d90-ff4d-4a07-9cd8-ca077870707e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:21.228315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f9d90-ff4d-4a07-9cd8-ca077870707e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:21.232888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b0022-ef51-4f4b-adeb-560c47a02838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:21.234135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0022-ef51-4f4b-adeb-560c47a02838 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:21.234580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0022-ef51-4f4b-adeb-560c47a02838 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:21.234869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0022-ef51-4f4b-adeb-560c47a02838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:21.250498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0022-ef51-4f4b-adeb-560c47a02838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:21.252287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0022-ef51-4f4b-adeb-560c47a02838 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:21.268470 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1165/2343] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:22:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:22:30.169251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc203f-12a0-48d2-af7e-f66997937b7c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:30.169801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc203f-12a0-48d2-af7e-f66997937b7c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:30.169897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc203f-12a0-48d2-af7e-f66997937b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:30.210820 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-3cfc203f-12a0-48d2-af7e-f66997937b7c None None] MFA Rules not processed for user `a21116ea64234269bba8b0369fb19b59`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:30.212720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3cfc203f-12a0-48d2-af7e-f66997937b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:30.218676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:30.316304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1179/2344] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:30.322521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b0022-ef51-4f4b-adeb-560c47a02838 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:30.329950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:30.336342 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b0022-ef51-4f4b-adeb-560c47a02838 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:30.363680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafdc5e-5557-464d-bb8f-6989601004c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:30.365224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafdc5e-5557-464d-bb8f-6989601004c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:30.367262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafdc5e-5557-464d-bb8f-6989601004c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:30.367829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafdc5e-5557-464d-bb8f-6989601004c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:30.368719 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafdc5e-5557-464d-bb8f-6989601004c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:30.370803 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafdc5e-5557-464d-bb8f-6989601004c9 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:30.435908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1166/2345] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:22:30 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:22:30.437635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc203f-12a0-48d2-af7e-f66997937b7c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:30.455311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc203f-12a0-48d2-af7e-f66997937b7c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:30.579898 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4724441-66ba-4e5f-a75f-ca1c228e8154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:30.580946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4724441-66ba-4e5f-a75f-ca1c228e8154 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:30.581240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4724441-66ba-4e5f-a75f-ca1c228e8154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:30.581521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4724441-66ba-4e5f-a75f-ca1c228e8154 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:30.582424 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4724441-66ba-4e5f-a75f-ca1c228e8154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:30.587980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4724441-66ba-4e5f-a75f-ca1c228e8154 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:30.633246 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1180/2346] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:22:30 2022] POST /identity/v3/users => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:22:30.635094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafdc5e-5557-464d-bb8f-6989601004c9 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:30.647767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafdc5e-5557-464d-bb8f-6989601004c9 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:30.652106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39875223-5a1e-4960-bb24-24903b93496e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:30.652872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39875223-5a1e-4960-bb24-24903b93496e admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:30.653225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39875223-5a1e-4960-bb24-24903b93496e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:30.653499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39875223-5a1e-4960-bb24-24903b93496e admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:30.654312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39875223-5a1e-4960-bb24-24903b93496e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:30.656010 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39875223-5a1e-4960-bb24-24903b93496e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:30.672162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1167/2347] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:22:30.673919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4724441-66ba-4e5f-a75f-ca1c228e8154 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:30.688965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4724441-66ba-4e5f-a75f-ca1c228e8154 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:30.695914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fe0b7-25a8-4b76-a8ca-0d3093d3db3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:30.697202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fe0b7-25a8-4b76-a8ca-0d3093d3db3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:30.697488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fe0b7-25a8-4b76-a8ca-0d3093d3db3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:30.697763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fe0b7-25a8-4b76-a8ca-0d3093d3db3c admin admin] PATH_INFO: `/v3/projects/1cdfcd3d2aa84226bc01f6400dd9754c/users/bd32f57db67345c8b524b624cca5c901/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:30.718049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fe0b7-25a8-4b76-a8ca-0d3093d3db3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cdfcd3d2aa84226bc01f6400dd9754c, user_id=bd32f57db67345c8b524b624cca5c901, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:30.720824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fe0b7-25a8-4b76-a8ca-0d3093d3db3c admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:30.744944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1181/2348] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:22:30 2022] PUT /identity/v3/projects/1cdfcd3d2aa84226bc01f6400dd9754c/users/bd32f57db67345c8b524b624cca5c901/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:22:30.749237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39875223-5a1e-4960-bb24-24903b93496e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:30.755734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220250c-9026-41cb-922c-ba080ebde3e7 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:30.756041 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220250c-9026-41cb-922c-ba080ebde3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:30.756405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220250c-9026-41cb-922c-ba080ebde3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:30.764820 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39875223-5a1e-4960-bb24-24903b93496e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:30.788038 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-5220250c-9026-41cb-922c-ba080ebde3e7 None None] MFA Rules not processed for user `86f2cda914b74f979746f4fb370eaae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:30.789684 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5220250c-9026-41cb-922c-ba080ebde3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:30.841667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b10a9-ff18-4f2e-8efe-4ff4f7785167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:30.845193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b10a9-ff18-4f2e-8efe-4ff4f7785167 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:30.845193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b10a9-ff18-4f2e-8efe-4ff4f7785167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:30.845193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b10a9-ff18-4f2e-8efe-4ff4f7785167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:30.845193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b10a9-ff18-4f2e-8efe-4ff4f7785167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:30.846057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b10a9-ff18-4f2e-8efe-4ff4f7785167 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:30.861107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1168/2349] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:22:30.862135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b10a9-ff18-4f2e-8efe-4ff4f7785167 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:30.874975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b10a9-ff18-4f2e-8efe-4ff4f7785167 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:30.880404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2989441-3240-4faf-b051-b59d0face6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:30.881529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2989441-3240-4faf-b051-b59d0face6ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:30.881903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2989441-3240-4faf-b051-b59d0face6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:30.882251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2989441-3240-4faf-b051-b59d0face6ff admin admin] PATH_INFO: `/v3/projects/1cdfcd3d2aa84226bc01f6400dd9754c/users/bd32f57db67345c8b524b624cca5c901/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:30.885231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2989441-3240-4faf-b051-b59d0face6ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cdfcd3d2aa84226bc01f6400dd9754c, user_id=bd32f57db67345c8b524b624cca5c901, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:30.888227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2989441-3240-4faf-b051-b59d0face6ff admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:30.889534 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1182/2350] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:30.893749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220250c-9026-41cb-922c-ba080ebde3e7 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:30.899669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:30.907903 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220250c-9026-41cb-922c-ba080ebde3e7 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:30.908364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1169/2351] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:22:30 2022] PUT /identity/v3/projects/1cdfcd3d2aa84226bc01f6400dd9754c/users/bd32f57db67345c8b524b624cca5c901/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:22:30.982958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6398f31-1eda-4d75-a9f2-7bed00bba82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:30.983797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6398f31-1eda-4d75-a9f2-7bed00bba82d admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:30.984182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6398f31-1eda-4d75-a9f2-7bed00bba82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:30.984865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6398f31-1eda-4d75-a9f2-7bed00bba82d admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:30.985324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6398f31-1eda-4d75-a9f2-7bed00bba82d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:30.987471 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6398f31-1eda-4d75-a9f2-7bed00bba82d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:31.019071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1183/2352] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:22:30 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:22:31.023020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2989441-3240-4faf-b051-b59d0face6ff admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:31.038278 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2989441-3240-4faf-b051-b59d0face6ff admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:31.165720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1276c66-6624-4f44-a1d1-da7e255f0676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:31.166430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1276c66-6624-4f44-a1d1-da7e255f0676 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:31.166722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1276c66-6624-4f44-a1d1-da7e255f0676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:31.167005 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1276c66-6624-4f44-a1d1-da7e255f0676 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:31.167707 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1276c66-6624-4f44-a1d1-da7e255f0676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:31.173897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1276c66-6624-4f44-a1d1-da7e255f0676 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:31.202689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1170/2353] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:22:31 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:22:31.206550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6398f31-1eda-4d75-a9f2-7bed00bba82d admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:31.218973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6398f31-1eda-4d75-a9f2-7bed00bba82d admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:31.225573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1c632-2974-445f-9c3b-68854c24f55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:31.226693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1c632-2974-445f-9c3b-68854c24f55f admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:31.227349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1c632-2974-445f-9c3b-68854c24f55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:31.228421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1c632-2974-445f-9c3b-68854c24f55f admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:31.229380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1c632-2974-445f-9c3b-68854c24f55f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:31.232572 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1c632-2974-445f-9c3b-68854c24f55f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:31.249036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1184/2354] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:22:31.253547 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1276c66-6624-4f44-a1d1-da7e255f0676 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:31.270700 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1276c66-6624-4f44-a1d1-da7e255f0676 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:31.276613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b530d-0581-4556-8e6b-7c74b5241e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:31.277546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b530d-0581-4556-8e6b-7c74b5241e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:31.277827 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b530d-0581-4556-8e6b-7c74b5241e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:31.278099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b530d-0581-4556-8e6b-7c74b5241e64 admin admin] PATH_INFO: `/v3/projects/e7c470fb14fb41878df38dde6131b126/users/1b657698fc114139b4e3655ca44291b6/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:31.295440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b530d-0581-4556-8e6b-7c74b5241e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c470fb14fb41878df38dde6131b126, user_id=1b657698fc114139b4e3655ca44291b6, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:31.298432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b530d-0581-4556-8e6b-7c74b5241e64 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:31.313973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1171/2355] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:22:31 2022] PUT /identity/v3/projects/e7c470fb14fb41878df38dde6131b126/users/1b657698fc114139b4e3655ca44291b6/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:22:31.325332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1c632-2974-445f-9c3b-68854c24f55f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:31.343561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1c632-2974-445f-9c3b-68854c24f55f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:31.453647 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a8f9e-0cd5-4df0-b7dd-6e1d25f045ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:31.454337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a8f9e-0cd5-4df0-b7dd-6e1d25f045ac admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:31.454623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a8f9e-0cd5-4df0-b7dd-6e1d25f045ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:31.454900 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a8f9e-0cd5-4df0-b7dd-6e1d25f045ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:31.455630 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a8f9e-0cd5-4df0-b7dd-6e1d25f045ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:31.458391 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a8f9e-0cd5-4df0-b7dd-6e1d25f045ac admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:31.474669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1185/2356] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:22:31.479187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b530d-0581-4556-8e6b-7c74b5241e64 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:31.494557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b530d-0581-4556-8e6b-7c74b5241e64 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:31.499947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7e709-3796-4319-8e93-e347361f38b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:31.501374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7e709-3796-4319-8e93-e347361f38b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:31.502175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7e709-3796-4319-8e93-e347361f38b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:31.502175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7e709-3796-4319-8e93-e347361f38b1 admin admin] PATH_INFO: `/v3/projects/e7c470fb14fb41878df38dde6131b126/users/1b657698fc114139b4e3655ca44291b6/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:31.507725 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7e709-3796-4319-8e93-e347361f38b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c470fb14fb41878df38dde6131b126, user_id=1b657698fc114139b4e3655ca44291b6, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:31.511185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7e709-3796-4319-8e93-e347361f38b1 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:31.526730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1172/2357] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:22:31 2022] PUT /identity/v3/projects/e7c470fb14fb41878df38dde6131b126/users/1b657698fc114139b4e3655ca44291b6/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:22:32.911943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea8f8c-0165-4daf-85d2-b33aeb4a0950 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:32.912972 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea8f8c-0165-4daf-85d2-b33aeb4a0950 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:32.913406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea8f8c-0165-4daf-85d2-b33aeb4a0950 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:32.916564 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1186/2358] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:22:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:22:32.925816 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4022b-f989-4ddc-9d5b-82e372c44b4f None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:32.926872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4022b-f989-4ddc-9d5b-82e372c44b4f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:32.927319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4022b-f989-4ddc-9d5b-82e372c44b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:32.934346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:33.082429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-85d4022b-f989-4ddc-9d5b-82e372c44b4f None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:33.085857 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-85d4022b-f989-4ddc-9d5b-82e372c44b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:33.347854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1173/2359] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:22:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:22:33.361657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea8f8c-0165-4daf-85d2-b33aeb4a0950 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:33.365083 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c8ea8f8c-0165-4daf-85d2-b33aeb4a0950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:33.467884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea8f8c-0165-4daf-85d2-b33aeb4a0950 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:33.478918 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2127d9d-4b3c-45ba-a550-29880b8c1410 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48586ff1269f49a3ae3d9f3e40c1b9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:33.479875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2127d9d-4b3c-45ba-a550-29880b8c1410 service glance] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:33.480389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2127d9d-4b3c-45ba-a550-29880b8c1410 service glance] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:33.480874 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2127d9d-4b3c-45ba-a550-29880b8c1410 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:33.504572 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2127d9d-4b3c-45ba-a550-29880b8c1410 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:33.506336 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2127d9d-4b3c-45ba-a550-29880b8c1410 service glance] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:33.544807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1187/2360] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:22:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:22:33.643080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36152f3e-8acc-45ee-b1e7-8f8d1db886ee None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:33.643323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36152f3e-8acc-45ee-b1e7-8f8d1db886ee None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:33.643621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36152f3e-8acc-45ee-b1e7-8f8d1db886ee None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:33.645031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1174/2361] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:22:33.651702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52201f34-28e1-4677-8613-41e2975ddd95 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:33.652929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52201f34-28e1-4677-8613-41e2975ddd95 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:33.653443 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52201f34-28e1-4677-8613-41e2975ddd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:33.677188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-52201f34-28e1-4677-8613-41e2975ddd95 None None] MFA Rules not processed for user `7e0797679c07454abc867cd7b30c8b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:33.679371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-52201f34-28e1-4677-8613-41e2975ddd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:33.694522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:33.782191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1188/2362] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:22:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:33.827817 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36152f3e-8acc-45ee-b1e7-8f8d1db886ee None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:33.853717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36152f3e-8acc-45ee-b1e7-8f8d1db886ee None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:33.866838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca140fc-86f3-4f9a-ad65-d5ffecd18b02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:33.867570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca140fc-86f3-4f9a-ad65-d5ffecd18b02 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:33.867861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca140fc-86f3-4f9a-ad65-d5ffecd18b02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:33.869621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca140fc-86f3-4f9a-ad65-d5ffecd18b02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:33.892482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca140fc-86f3-4f9a-ad65-d5ffecd18b02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:33.894026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca140fc-86f3-4f9a-ad65-d5ffecd18b02 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:33.928509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1175/2363] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:22:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:22:34.152858 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af2744-7b7b-40f0-a683-22de423f95b8 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:34.153331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af2744-7b7b-40f0-a683-22de423f95b8 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:34.154129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af2744-7b7b-40f0-a683-22de423f95b8 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:34.155365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1189/2364] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:22:34.162275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f290e-47d2-4a3c-973b-a571a8f65cee None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:34.162967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f290e-47d2-4a3c-973b-a571a8f65cee None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:34.163585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f290e-47d2-4a3c-973b-a571a8f65cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:34.214798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-e61f290e-47d2-4a3c-973b-a571a8f65cee None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:34.216458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e61f290e-47d2-4a3c-973b-a571a8f65cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:34.232859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1176/2365] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:22:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:34.607916 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af2744-7b7b-40f0-a683-22de423f95b8 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:34.611373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-50af2744-7b7b-40f0-a683-22de423f95b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:34.646277 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af2744-7b7b-40f0-a683-22de423f95b8 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:34.650606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62678a-f911-4776-950b-681645855478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:34.651370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62678a-f911-4776-950b-681645855478 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:34.651664 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62678a-f911-4776-950b-681645855478 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:34.651948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62678a-f911-4776-950b-681645855478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:34.676975 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62678a-f911-4776-950b-681645855478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:34.678595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62678a-f911-4776-950b-681645855478 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:34.706478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1190/2366] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:22:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:22:37.740694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fca3ce-9a56-4f78-bd6b-b5ea90d1a81d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:37.741804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fca3ce-9a56-4f78-bd6b-b5ea90d1a81d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:37.742351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fca3ce-9a56-4f78-bd6b-b5ea90d1a81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:37.778409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-a0fca3ce-9a56-4f78-bd6b-b5ea90d1a81d None None] MFA Rules not processed for user `bd32f57db67345c8b524b624cca5c901`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:37.780541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a0fca3ce-9a56-4f78-bd6b-b5ea90d1a81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:37.891794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1177/2367] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:37.893749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62678a-f911-4776-950b-681645855478 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:37.906913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62678a-f911-4776-950b-681645855478 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:37.911631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0897c01-39a8-4395-a53b-08da80c15f3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48586ff1269f49a3ae3d9f3e40c1b9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:37.912776 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0897c01-39a8-4395-a53b-08da80c15f3c service glance] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:37.913381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0897c01-39a8-4395-a53b-08da80c15f3c service glance] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:37.914218 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0897c01-39a8-4395-a53b-08da80c15f3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:38.011553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0897c01-39a8-4395-a53b-08da80c15f3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:38.014025 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0897c01-39a8-4395-a53b-08da80c15f3c service glance] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:38.056710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1191/2368] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:22:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:22:38.270044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b63553-8ec9-4691-b3ca-94e6520703ea None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:38.270340 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b63553-8ec9-4691-b3ca-94e6520703ea None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:38.270622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b63553-8ec9-4691-b3ca-94e6520703ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:38.302575 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-09b63553-8ec9-4691-b3ca-94e6520703ea None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:38.303815 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-09b63553-8ec9-4691-b3ca-94e6520703ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:38.325596 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:38.480210 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1178/2369] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:22:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:38.488910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0897c01-39a8-4395-a53b-08da80c15f3c service glance] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:38.490708 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c0897c01-39a8-4395-a53b-08da80c15f3c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:38.710530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0897c01-39a8-4395-a53b-08da80c15f3c service glance] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:38.735093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d088a07-9fe5-476c-901d-bcaede863bf5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:38.736013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d088a07-9fe5-476c-901d-bcaede863bf5 service swift] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:38.736719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d088a07-9fe5-476c-901d-bcaede863bf5 service swift] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:38.737365 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d088a07-9fe5-476c-901d-bcaede863bf5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:38.762652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d088a07-9fe5-476c-901d-bcaede863bf5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:38.765141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d088a07-9fe5-476c-901d-bcaede863bf5 service swift] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:38.791762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1192/2370] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:22:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:22:39.542001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59054-5134-44e9-9261-81939133247a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:39.542488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59054-5134-44e9-9261-81939133247a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:39.542585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59054-5134-44e9-9261-81939133247a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:39.590830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-32a59054-5134-44e9-9261-81939133247a None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:39.592100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-32a59054-5134-44e9-9261-81939133247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:39.611996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1179/2371] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:22:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:39.637012 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d088a07-9fe5-476c-901d-bcaede863bf5 service swift] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:39.661592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d088a07-9fe5-476c-901d-bcaede863bf5 service swift] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:39.701393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae6bf2-76b9-4b46-87ac-780d65557dd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:39.702357 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae6bf2-76b9-4b46-87ac-780d65557dd3 service swift] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:39.702854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae6bf2-76b9-4b46-87ac-780d65557dd3 service swift] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:39.703367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae6bf2-76b9-4b46-87ac-780d65557dd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:39.725125 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae6bf2-76b9-4b46-87ac-780d65557dd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:39.739331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae6bf2-76b9-4b46-87ac-780d65557dd3 service swift] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:39.766649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1193/2372] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:22:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:22:41.147926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8311e93-53c3-49bc-9439-106329915e02 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:41.148733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8311e93-53c3-49bc-9439-106329915e02 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:41.148733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8311e93-53c3-49bc-9439-106329915e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:41.264169 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c8311e93-53c3-49bc-9439-106329915e02 None None] MFA Rules not processed for user `1b657698fc114139b4e3655ca44291b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:41.265744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c8311e93-53c3-49bc-9439-106329915e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:41.457694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1180/2373] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:41.468746 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae6bf2-76b9-4b46-87ac-780d65557dd3 service swift] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:41.470336 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7dae6bf2-76b9-4b46-87ac-780d65557dd3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:41.529771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae6bf2-76b9-4b46-87ac-780d65557dd3 service swift] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:41.558607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664af87-5425-4f60-90cc-0494b2ba2b52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:41.561504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664af87-5425-4f60-90cc-0494b2ba2b52 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:41.562950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664af87-5425-4f60-90cc-0494b2ba2b52 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:41.563993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664af87-5425-4f60-90cc-0494b2ba2b52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:41.664802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:22:41.758740 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664af87-5425-4f60-90cc-0494b2ba2b52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:41.761108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664af87-5425-4f60-90cc-0494b2ba2b52 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:41.788623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1194/2374] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:22:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:22:45.269203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41504c3d-8c9f-4905-9043-745a0fe45508 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:45.271073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41504c3d-8c9f-4905-9043-745a0fe45508 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:45.271706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41504c3d-8c9f-4905-9043-745a0fe45508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:45.330401 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-41504c3d-8c9f-4905-9043-745a0fe45508 None None] MFA Rules not processed for user `86f2cda914b74f979746f4fb370eaae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:45.331643 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-41504c3d-8c9f-4905-9043-745a0fe45508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:45.358443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1181/2375] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:46.383171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664af87-5425-4f60-90cc-0494b2ba2b52 service nova] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:22:46.385863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-a664af87-5425-4f60-90cc-0494b2ba2b52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:46.661758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664af87-5425-4f60-90cc-0494b2ba2b52 service nova] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:22:46.687334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70596d7f-0845-46c0-ac58-7b95bc166332 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8cf02a52d354517b363c0aaf477550c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:22:46.688013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596d7f-0845-46c0-ac58-7b95bc166332 service placement] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:46.688410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596d7f-0845-46c0-ac58-7b95bc166332 service placement] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:46.688711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596d7f-0845-46c0-ac58-7b95bc166332 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:46.700985 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-70596d7f-0845-46c0-ac58-7b95bc166332 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:46.733353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70596d7f-0845-46c0-ac58-7b95bc166332 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:22:46.735474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70596d7f-0845-46c0-ac58-7b95bc166332 service placement] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:22:46.752226 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1195/2376] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:22:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:22:54.819646 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f01b34-4a7a-42e6-8454-e582df94fc80 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:22:54.820450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f01b34-4a7a-42e6-8454-e582df94fc80 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:22:54.820985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f01b34-4a7a-42e6-8454-e582df94fc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:22:54.846935 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-b8f01b34-4a7a-42e6-8454-e582df94fc80 None None] MFA Rules not processed for user `a21116ea64234269bba8b0369fb19b59`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:22:54.848197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b8f01b34-4a7a-42e6-8454-e582df94fc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:22:54.856146 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1182/2377] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:22:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:22:56.742786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=57 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:23:02.977235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804c1e1-9d0d-4b7e-966a-0b1b78d4acaf None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:23:02.978401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804c1e1-9d0d-4b7e-966a-0b1b78d4acaf None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:23:02.978518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804c1e1-9d0d-4b7e-966a-0b1b78d4acaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:23:03.042372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-d804c1e1-9d0d-4b7e-966a-0b1b78d4acaf None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:23:03.043680 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d804c1e1-9d0d-4b7e-966a-0b1b78d4acaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:23:03.062324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1196/2378] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:23:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:23:03.088287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f01b34-4a7a-42e6-8454-e582df94fc80 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:23:03.111902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f01b34-4a7a-42e6-8454-e582df94fc80 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:23:03.117461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217a22a-e06b-48f3-a19a-8ef6665555e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:23:03.118178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217a22a-e06b-48f3-a19a-8ef6665555e4 service swift] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:23:03.118459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217a22a-e06b-48f3-a19a-8ef6665555e4 service swift] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:23:03.118811 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217a22a-e06b-48f3-a19a-8ef6665555e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:23:03.127025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217a22a-e06b-48f3-a19a-8ef6665555e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:23:03.129025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217a22a-e06b-48f3-a19a-8ef6665555e4 service swift] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:23:03.140641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1183/2379] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:23:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:23:13.059205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=18 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:23:13.134694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=72 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:23:20.939492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164defca-7a36-4704-9b16-423d1b7d2f98 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:23:20.940053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164defca-7a36-4704-9b16-423d1b7d2f98 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:23:20.940053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164defca-7a36-4704-9b16-423d1b7d2f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:23:20.961011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-164defca-7a36-4704-9b16-423d1b7d2f98 None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:23:20.962591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-164defca-7a36-4704-9b16-423d1b7d2f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:23:20.971132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1197/2380] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jun 17 07:23:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:23:20.983138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217a22a-e06b-48f3-a19a-8ef6665555e4 service swift] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:23:20.994069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217a22a-e06b-48f3-a19a-8ef6665555e4 service swift] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:23:20.999096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3f588-65df-4d7b-9b65-5cc8702304e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:23:20.999791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3f588-65df-4d7b-9b65-5cc8702304e3 service swift] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:23:21.000155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3f588-65df-4d7b-9b65-5cc8702304e3 service swift] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:23:21.000417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3f588-65df-4d7b-9b65-5cc8702304e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:23:21.009038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3f588-65df-4d7b-9b65-5cc8702304e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:23:21.010624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3f588-65df-4d7b-9b65-5cc8702304e3 service swift] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:23:21.018589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1184/2381] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:23:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:23:30.953570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:23:31.015524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=6 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:23:31.803695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a75ae-dd34-4595-bce5-3420464f488d None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:23:31.804003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a75ae-dd34-4595-bce5-3420464f488d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:23:31.804376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a75ae-dd34-4595-bce5-3420464f488d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:23:31.825425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-744a75ae-dd34-4595-bce5-3420464f488d None None] MFA Rules not processed for user `7615cf807ce746ada15efa0ff1cb80d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:23:31.826754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-744a75ae-dd34-4595-bce5-3420464f488d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:23:31.889763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1198/2382] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:23:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:23:37.125224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3f588-65df-4d7b-9b65-5cc8702304e3 service swift] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:23:37.134850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3f588-65df-4d7b-9b65-5cc8702304e3 service swift] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:23:37.138404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d4fe4-a5c0-46e3-9366-ab41cb7895ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:23:37.139035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d4fe4-a5c0-46e3-9366-ab41cb7895ed service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:23:37.139324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d4fe4-a5c0-46e3-9366-ab41cb7895ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:23:37.139650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d4fe4-a5c0-46e3-9366-ab41cb7895ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:23:37.148511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d4fe4-a5c0-46e3-9366-ab41cb7895ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:23:37.150474 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d4fe4-a5c0-46e3-9366-ab41cb7895ed service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:23:37.159770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1185/2383] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:23:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:23:39.816472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a75ae-dd34-4595-bce5-3420464f488d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:23:39.827122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a75ae-dd34-4595-bce5-3420464f488d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:23:39.831072 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b527ea-655e-4176-957d-708125fb0d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:23:39.831738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b527ea-655e-4176-957d-708125fb0d55 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:23:39.832024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b527ea-655e-4176-957d-708125fb0d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:23:39.832511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b527ea-655e-4176-957d-708125fb0d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:23:39.843497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b527ea-655e-4176-957d-708125fb0d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:23:39.845246 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b527ea-655e-4176-957d-708125fb0d55 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:23:39.864296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1199/2384] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:23:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:23:47.155653 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=6 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:23:47.703199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bc456-af90-4e18-a531-611a4f89e450 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:23:47.703494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bc456-af90-4e18-a531-611a4f89e450 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:23:47.703778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bc456-af90-4e18-a531-611a4f89e450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:23:47.721405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-697bc456-af90-4e18-a531-611a4f89e450 None None] MFA Rules not processed for user `c11f44eb67cd4923a840171833bb572a`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:23:47.722690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-697bc456-af90-4e18-a531-611a4f89e450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:23:47.779725 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1186/2385] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:23:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:23:49.852662 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=31 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:23:57.772374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=25 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:24:30.579452 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95253c55-06f3-4613-b8b1-fa2a7dd2398c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:30.579967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95253c55-06f3-4613-b8b1-fa2a7dd2398c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:30.580121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95253c55-06f3-4613-b8b1-fa2a7dd2398c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:30.597153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-95253c55-06f3-4613-b8b1-fa2a7dd2398c None None] MFA Rules not processed for user `31e94cc1c6a743ce898424ad7ba26cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:24:30.598355 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-95253c55-06f3-4613-b8b1-fa2a7dd2398c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:30.651473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1200/2386] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:24:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:24:35.369019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bc456-af90-4e18-a531-611a4f89e450 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:35.370547 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-697bc456-af90-4e18-a531-611a4f89e450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:35.451582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bc456-af90-4e18-a531-611a4f89e450 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:35.457251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c52d1-8934-4785-8a15-6c46024f32ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:35.458162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c52d1-8934-4785-8a15-6c46024f32ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:35.458446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c52d1-8934-4785-8a15-6c46024f32ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:35.458719 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c52d1-8934-4785-8a15-6c46024f32ba admin admin] PATH_INFO: `/v3/users/7615cf807ce746ada15efa0ff1cb80d7` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:35.460090 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c52d1-8934-4785-8a15-6c46024f32ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=7615cf807ce746ada15efa0ff1cb80d7)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:35.462211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c52d1-8934-4785-8a15-6c46024f32ba admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:35.510311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e64c52d1-8934-4785-8a15-6c46024f32ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7615cf807ce746ada15efa0ff1cb80d7'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:35.510972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e64c52d1-8934-4785-8a15-6c46024f32ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7615cf807ce746ada15efa0ff1cb80d7'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:35.517826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e64c52d1-8934-4785-8a15-6c46024f32ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7615cf807ce746ada15efa0ff1cb80d7'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:35.529711 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e64c52d1-8934-4785-8a15-6c46024f32ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7615cf807ce746ada15efa0ff1cb80d7'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:35.530409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e64c52d1-8934-4785-8a15-6c46024f32ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7615cf807ce746ada15efa0ff1cb80d7'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:35.544913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1187/2387] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:24:35 2022] DELETE /identity/v3/users/7615cf807ce746ada15efa0ff1cb80d7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:24:35.724150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95253c55-06f3-4613-b8b1-fa2a7dd2398c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:35.743988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95253c55-06f3-4613-b8b1-fa2a7dd2398c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:35.807036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd61e7-1eb1-4484-a01b-a74a5cd87283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:35.807830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd61e7-1eb1-4484-a01b-a74a5cd87283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:35.808157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd61e7-1eb1-4484-a01b-a74a5cd87283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:35.808444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd61e7-1eb1-4484-a01b-a74a5cd87283 admin admin] PATH_INFO: `/v3/projects/71a5393056b74f1e871bb50775a40554` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:35.809806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd61e7-1eb1-4484-a01b-a74a5cd87283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a5393056b74f1e871bb50775a40554)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:35.812320 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd61e7-1eb1-4484-a01b-a74a5cd87283 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:35.867739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-eadd61e7-1eb1-4484-a01b-a74a5cd87283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a5393056b74f1e871bb50775a40554', 'request_id': 'req-eadd61e7-1eb1-4484-a01b-a74a5cd87283'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:35.874288 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-eadd61e7-1eb1-4484-a01b-a74a5cd87283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a5393056b74f1e871bb50775a40554', 'request_id': 'req-eadd61e7-1eb1-4484-a01b-a74a5cd87283'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:35.880733 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-eadd61e7-1eb1-4484-a01b-a74a5cd87283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a5393056b74f1e871bb50775a40554', 'request_id': 'req-eadd61e7-1eb1-4484-a01b-a74a5cd87283'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:35.895876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-eadd61e7-1eb1-4484-a01b-a74a5cd87283 admin admin] The token cache is being invalidate because project 71a5393056b74f1e871bb50775a40554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:24:35.896187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-eadd61e7-1eb1-4484-a01b-a74a5cd87283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:35.896731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-eadd61e7-1eb1-4484-a01b-a74a5cd87283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:35.898304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1201/2388] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:24:35 2022] DELETE /identity/v3/projects/71a5393056b74f1e871bb50775a40554 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:24:35.903143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7f06b-4dbd-4242-8cf0-b7e1e9a63940 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:35.903437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7f06b-4dbd-4242-8cf0-b7e1e9a63940 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:35.903720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7f06b-4dbd-4242-8cf0-b7e1e9a63940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:35.924141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-cbe7f06b-4dbd-4242-8cf0-b7e1e9a63940 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:24:35.925336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-cbe7f06b-4dbd-4242-8cf0-b7e1e9a63940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:35.992384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1188/2389] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:24:35.999858 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec86561-32b2-4d77-a03a-25ef269dbbe8 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:36.000180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec86561-32b2-4d77-a03a-25ef269dbbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:36.000466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec86561-32b2-4d77-a03a-25ef269dbbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:36.018362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-8ec86561-32b2-4d77-a03a-25ef269dbbe8 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:24:36.019570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8ec86561-32b2-4d77-a03a-25ef269dbbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:36.028124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1202/2390] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:24:36.032637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7f06b-4dbd-4242-8cf0-b7e1e9a63940 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:36.042658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7f06b-4dbd-4242-8cf0-b7e1e9a63940 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:36.047437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b1a61-dd9d-4696-aa5e-13ba30a2acdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:36.048132 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b1a61-dd9d-4696-aa5e-13ba30a2acdc admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:36.048427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b1a61-dd9d-4696-aa5e-13ba30a2acdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:36.048709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b1a61-dd9d-4696-aa5e-13ba30a2acdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:36.049457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b1a61-dd9d-4696-aa5e-13ba30a2acdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:24:36.049750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b1a61-dd9d-4696-aa5e-13ba30a2acdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:36.051345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b1a61-dd9d-4696-aa5e-13ba30a2acdc admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:36.064178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1189/2391] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:24:36.068108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec86561-32b2-4d77-a03a-25ef269dbbe8 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:36.078144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec86561-32b2-4d77-a03a-25ef269dbbe8 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:36.082872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d3d9d-5093-4494-b92b-0b09fd323f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:36.083514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d3d9d-5093-4494-b92b-0b09fd323f19 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:36.083801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d3d9d-5093-4494-b92b-0b09fd323f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:36.084108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d3d9d-5093-4494-b92b-0b09fd323f19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:36.084795 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d3d9d-5093-4494-b92b-0b09fd323f19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:36.086711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d3d9d-5093-4494-b92b-0b09fd323f19 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:36.113808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1203/2392] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:24:36 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:24:36.117877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b1a61-dd9d-4696-aa5e-13ba30a2acdc admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:36.127702 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b1a61-dd9d-4696-aa5e-13ba30a2acdc admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:36.131306 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:24:36.189239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb8c29-e3ad-447a-8c32-7060cc4ca236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:36.189919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb8c29-e3ad-447a-8c32-7060cc4ca236 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:36.190204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb8c29-e3ad-447a-8c32-7060cc4ca236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:36.190483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb8c29-e3ad-447a-8c32-7060cc4ca236 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:36.191173 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb8c29-e3ad-447a-8c32-7060cc4ca236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:36.193159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb8c29-e3ad-447a-8c32-7060cc4ca236 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:36.214044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1190/2393] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:24:36 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:24:36.218479 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d3d9d-5093-4494-b92b-0b09fd323f19 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:36.228273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d3d9d-5093-4494-b92b-0b09fd323f19 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:36.234961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044096f3-5a2d-4972-9bb4-af57c77049cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:36.235930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044096f3-5a2d-4972-9bb4-af57c77049cc admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:36.236368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044096f3-5a2d-4972-9bb4-af57c77049cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:36.236754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044096f3-5a2d-4972-9bb4-af57c77049cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:36.237598 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044096f3-5a2d-4972-9bb4-af57c77049cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:36.239940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044096f3-5a2d-4972-9bb4-af57c77049cc admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:36.241473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:24:36.256391 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1204/2394] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:24:36.257576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb8c29-e3ad-447a-8c32-7060cc4ca236 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:36.269591 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb8c29-e3ad-447a-8c32-7060cc4ca236 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:36.274008 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd528a17-7884-4650-8763-6ffa3b43f168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:36.274755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd528a17-7884-4650-8763-6ffa3b43f168 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:36.275040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd528a17-7884-4650-8763-6ffa3b43f168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:36.275318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd528a17-7884-4650-8763-6ffa3b43f168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:36.276268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd528a17-7884-4650-8763-6ffa3b43f168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:36.278302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd528a17-7884-4650-8763-6ffa3b43f168 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:36.290066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1191/2395] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:24:36.294565 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044096f3-5a2d-4972-9bb4-af57c77049cc admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:36.306444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044096f3-5a2d-4972-9bb4-af57c77049cc admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:36.311728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f05a7-3235-47aa-b8e6-211178b9417f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:36.312660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f05a7-3235-47aa-b8e6-211178b9417f admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:36.312945 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f05a7-3235-47aa-b8e6-211178b9417f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:36.313225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f05a7-3235-47aa-b8e6-211178b9417f admin admin] PATH_INFO: `/v3/projects/31ba39f47f7c40cd9f95168cdb513777/users/b311fcc412e347b3b8c186ca006acc12/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:36.328943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f05a7-3235-47aa-b8e6-211178b9417f admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ba39f47f7c40cd9f95168cdb513777, user_id=b311fcc412e347b3b8c186ca006acc12, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:36.331553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f05a7-3235-47aa-b8e6-211178b9417f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:36.344249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1205/2396] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:24:36 2022] PUT /identity/v3/projects/31ba39f47f7c40cd9f95168cdb513777/users/b311fcc412e347b3b8c186ca006acc12/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:24:36.355096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ba3d0-168f-40db-b850-93b14b801afa None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:36.355440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ba3d0-168f-40db-b850-93b14b801afa None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:36.355723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ba3d0-168f-40db-b850-93b14b801afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:36.384485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-6a0ba3d0-168f-40db-b850-93b14b801afa None None] MFA Rules not processed for user `b311fcc412e347b3b8c186ca006acc12`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:24:36.385742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6a0ba3d0-168f-40db-b850-93b14b801afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:36.450958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1192/2397] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:24:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:24:36.462207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f05a7-3235-47aa-b8e6-211178b9417f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:36.463897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e65f05a7-3235-47aa-b8e6-211178b9417f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:36.540401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f05a7-3235-47aa-b8e6-211178b9417f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:36.544357 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541c60f-71fb-409a-b986-346003bd29d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:36.545002 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541c60f-71fb-409a-b986-346003bd29d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:36.545288 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541c60f-71fb-409a-b986-346003bd29d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:36.545595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541c60f-71fb-409a-b986-346003bd29d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:36.553895 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541c60f-71fb-409a-b986-346003bd29d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:36.555336 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541c60f-71fb-409a-b986-346003bd29d3 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:36.564768 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1206/2398] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:24:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:24:38.340443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ba3d0-168f-40db-b850-93b14b801afa None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:38.341891 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6a0ba3d0-168f-40db-b850-93b14b801afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:38.406409 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ba3d0-168f-40db-b850-93b14b801afa None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:38.410303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1a610-1216-444e-8233-bc65cec0e2af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:38.410945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1a610-1216-444e-8233-bc65cec0e2af service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:38.411227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1a610-1216-444e-8233-bc65cec0e2af service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:38.411528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1a610-1216-444e-8233-bc65cec0e2af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:38.413322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-dad1a610-1216-444e-8233-bc65cec0e2af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:38.446369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:24:38.488963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1a610-1216-444e-8233-bc65cec0e2af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:38.490432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1a610-1216-444e-8233-bc65cec0e2af service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:38.501927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1193/2399] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:24:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:24:39.738757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541c60f-71fb-409a-b986-346003bd29d3 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:39.740263 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b541c60f-71fb-409a-b986-346003bd29d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:39.752966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541c60f-71fb-409a-b986-346003bd29d3 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:39.757323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bc517-9a4a-4169-9cd3-0bbb5f42ef74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:39.757972 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bc517-9a4a-4169-9cd3-0bbb5f42ef74 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:39.758260 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bc517-9a4a-4169-9cd3-0bbb5f42ef74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:39.758542 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bc517-9a4a-4169-9cd3-0bbb5f42ef74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:39.760344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-230bc517-9a4a-4169-9cd3-0bbb5f42ef74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:39.774901 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bc517-9a4a-4169-9cd3-0bbb5f42ef74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:39.776438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bc517-9a4a-4169-9cd3-0bbb5f42ef74 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:39.787838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1207/2400] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:24:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:24:44.311969 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1a610-1216-444e-8233-bc65cec0e2af service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:44.313858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-dad1a610-1216-444e-8233-bc65cec0e2af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:44.325634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1a610-1216-444e-8233-bc65cec0e2af service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:44.329424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5cfcb-4c5c-4f37-8c37-c8b73fdb368f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:44.330101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5cfcb-4c5c-4f37-8c37-c8b73fdb368f service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:44.330387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5cfcb-4c5c-4f37-8c37-c8b73fdb368f service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:44.330663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5cfcb-4c5c-4f37-8c37-c8b73fdb368f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:44.332455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a1f5cfcb-4c5c-4f37-8c37-c8b73fdb368f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:44.400842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5cfcb-4c5c-4f37-8c37-c8b73fdb368f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:44.402432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5cfcb-4c5c-4f37-8c37-c8b73fdb368f service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:44.412015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1194/2401] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:24:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:24:44.662807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bc517-9a4a-4169-9cd3-0bbb5f42ef74 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:44.750913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bc517-9a4a-4169-9cd3-0bbb5f42ef74 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:44.755798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:44.756454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:44.756753 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:44.757024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] PATH_INFO: `/v3/users/b311fcc412e347b3b8c186ca006acc12` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:44.758308 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b311fcc412e347b3b8c186ca006acc12)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:44.760436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:44.816795 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:24:44.819927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b311fcc412e347b3b8c186ca006acc12'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:44.831421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b311fcc412e347b3b8c186ca006acc12'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:44.836450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b311fcc412e347b3b8c186ca006acc12'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:44.851354 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b311fcc412e347b3b8c186ca006acc12'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:44.852093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b311fcc412e347b3b8c186ca006acc12'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:44.867869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1208/2402] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:24:44 2022] DELETE /identity/v3/users/b311fcc412e347b3b8c186ca006acc12 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:24:44.869927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752feec6-eddd-4c41-a432-cf1a268ab955 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:44.870289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752feec6-eddd-4c41-a432-cf1a268ab955 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:44.870629 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752feec6-eddd-4c41-a432-cf1a268ab955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:44.889182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-752feec6-eddd-4c41-a432-cf1a268ab955 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:24:44.891093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-752feec6-eddd-4c41-a432-cf1a268ab955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:44.962188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1195/2403] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:24:44.972218 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:44.973705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:45.033009 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a28f1-b850-4dcb-8703-82b37b80d5b6 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:45.036744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3ef30-1940-42b2-9136-1ab9afa4e3f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:45.037395 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3ef30-1940-42b2-9136-1ab9afa4e3f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:45.037679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3ef30-1940-42b2-9136-1ab9afa4e3f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:45.037958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3ef30-1940-42b2-9136-1ab9afa4e3f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:45.048371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3ef30-1940-42b2-9136-1ab9afa4e3f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:45.049894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3ef30-1940-42b2-9136-1ab9afa4e3f1 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:45.060270 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1209/2404] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:24:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:24:45.351842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752feec6-eddd-4c41-a432-cf1a268ab955 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:45.353472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-752feec6-eddd-4c41-a432-cf1a268ab955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:45.368712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752feec6-eddd-4c41-a432-cf1a268ab955 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:45.375109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:45.376253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:45.376633 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:45.376802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] PATH_INFO: `/v3/projects/31ba39f47f7c40cd9f95168cdb513777` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:45.378059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ba39f47f7c40cd9f95168cdb513777)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:45.380189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:45.431807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ba39f47f7c40cd9f95168cdb513777', 'request_id': 'req-3382e60f-5c67-4089-af66-895cb18fbfdf'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:45.433569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:24:45.440311 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ba39f47f7c40cd9f95168cdb513777', 'request_id': 'req-3382e60f-5c67-4089-af66-895cb18fbfdf'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:45.447408 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ba39f47f7c40cd9f95168cdb513777', 'request_id': 'req-3382e60f-5c67-4089-af66-895cb18fbfdf'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:45.458225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] The token cache is being invalidate because project 31ba39f47f7c40cd9f95168cdb513777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:24:45.458630 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:45.459382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:24:45.461249 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1196/2405] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:24:45 2022] DELETE /identity/v3/projects/31ba39f47f7c40cd9f95168cdb513777 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:24:53.517942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95151692-23c9-44b6-9c96-f1ffb3955079 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:53.518448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95151692-23c9-44b6-9c96-f1ffb3955079 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:53.518568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95151692-23c9-44b6-9c96-f1ffb3955079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:53.537110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-95151692-23c9-44b6-9c96-f1ffb3955079 None None] MFA Rules not processed for user `b3ca63f8f8484ddf9d7c72749ba7f482`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:24:53.538318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-95151692-23c9-44b6-9c96-f1ffb3955079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:53.601762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1210/2406] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:24:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:24:54.314711 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:54.316204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:54.408960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382e60f-5c67-4089-af66-895cb18fbfdf admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:54.414779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8489fd-6f8b-4547-b7d3-c61fd69ca84e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8cf02a52d354517b363c0aaf477550c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:54.415519 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8489fd-6f8b-4547-b7d3-c61fd69ca84e service placement] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:54.415869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8489fd-6f8b-4547-b7d3-c61fd69ca84e service placement] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:54.416251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8489fd-6f8b-4547-b7d3-c61fd69ca84e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:54.418453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-2c8489fd-6f8b-4547-b7d3-c61fd69ca84e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:54.431030 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8489fd-6f8b-4547-b7d3-c61fd69ca84e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:54.432683 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8489fd-6f8b-4547-b7d3-c61fd69ca84e service placement] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:54.443321 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1197/2407] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:24:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:24:56.719368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95151692-23c9-44b6-9c96-f1ffb3955079 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:24:56.721031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-95151692-23c9-44b6-9c96-f1ffb3955079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:56.780973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95151692-23c9-44b6-9c96-f1ffb3955079 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:24:56.785228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35309c60-c167-4a9f-af7c-113b8c93ee04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:24:56.785887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35309c60-c167-4a9f-af7c-113b8c93ee04 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:24:56.786171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35309c60-c167-4a9f-af7c-113b8c93ee04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:24:56.786442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35309c60-c167-4a9f-af7c-113b8c93ee04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:24:56.789449 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-35309c60-c167-4a9f-af7c-113b8c93ee04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:24:56.815617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:24:56.860785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35309c60-c167-4a9f-af7c-113b8c93ee04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:24:56.862840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35309c60-c167-4a9f-af7c-113b8c93ee04 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:24:56.883892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1211/2408] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:24:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:25:04.438925 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=38 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:05.271648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8489fd-6f8b-4547-b7d3-c61fd69ca84e service placement] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:05.273101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-2c8489fd-6f8b-4547-b7d3-c61fd69ca84e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:05.329471 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8489fd-6f8b-4547-b7d3-c61fd69ca84e service placement] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:05.333040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b002b50-7d57-488e-a859-bd7cfe5318ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:05.333708 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b002b50-7d57-488e-a859-bd7cfe5318ff service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:05.333992 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b002b50-7d57-488e-a859-bd7cfe5318ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:05.334274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b002b50-7d57-488e-a859-bd7cfe5318ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:05.336050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-2b002b50-7d57-488e-a859-bd7cfe5318ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:05.345930 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b002b50-7d57-488e-a859-bd7cfe5318ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:05.347362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b002b50-7d57-488e-a859-bd7cfe5318ff service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:05.356801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1198/2409] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:25:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:25:06.875098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=17 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:07.678412 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35309c60-c167-4a9f-af7c-113b8c93ee04 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:07.679830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-35309c60-c167-4a9f-af7c-113b8c93ee04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:07.692558 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35309c60-c167-4a9f-af7c-113b8c93ee04 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:07.696164 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ff692-3bbb-413e-a2a7-1b8f4772f57e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:07.696817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ff692-3bbb-413e-a2a7-1b8f4772f57e service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:07.697101 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ff692-3bbb-413e-a2a7-1b8f4772f57e service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:07.697376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ff692-3bbb-413e-a2a7-1b8f4772f57e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:07.699148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-4c9ff692-3bbb-413e-a2a7-1b8f4772f57e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:07.765030 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ff692-3bbb-413e-a2a7-1b8f4772f57e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:07.766476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ff692-3bbb-413e-a2a7-1b8f4772f57e service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:07.776101 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1212/2410] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:25:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:25:11.316427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdace0d-3770-4c85-89af-5f713dc4b4d5 None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:11.316907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdace0d-3770-4c85-89af-5f713dc4b4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:11.317001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdace0d-3770-4c85-89af-5f713dc4b4d5 None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:11.318520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1199/2411] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:25:11.327344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d680625-9de0-4f88-95ba-6298141c33b5 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:11.328267 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d680625-9de0-4f88-95ba-6298141c33b5 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:11.328596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d680625-9de0-4f88-95ba-6298141c33b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:11.365514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-4d680625-9de0-4f88-95ba-6298141c33b5 None None] MFA Rules not processed for user `b8cf02a52d354517b363c0aaf477550c`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:11.366710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-4d680625-9de0-4f88-95ba-6298141c33b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:11.377932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1213/2412] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:25:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:25:11.395568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdace0d-3770-4c85-89af-5f713dc4b4d5 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:11.397691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-afdace0d-3770-4c85-89af-5f713dc4b4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:11.398817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d680625-9de0-4f88-95ba-6298141c33b5 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:11.410129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d680625-9de0-4f88-95ba-6298141c33b5 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:11.419713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdace0d-3770-4c85-89af-5f713dc4b4d5 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:11.422451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33252db4-bc4a-4baa-a45c-fb44f4dbcb75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8cf02a52d354517b363c0aaf477550c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:11.423105 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33252db4-bc4a-4baa-a45c-fb44f4dbcb75 service placement] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:11.423394 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33252db4-bc4a-4baa-a45c-fb44f4dbcb75 service placement] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:11.423675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33252db4-bc4a-4baa-a45c-fb44f4dbcb75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:11.426424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2726c-541c-4c84-84ad-1737b7da2e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:11.427379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2726c-541c-4c84-84ad-1737b7da2e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:11.427738 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2726c-541c-4c84-84ad-1737b7da2e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:11.428176 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2726c-541c-4c84-84ad-1737b7da2e39 admin admin] PATH_INFO: `/v3/users/c11f44eb67cd4923a840171833bb572a` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:11.429463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2726c-541c-4c84-84ad-1737b7da2e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c11f44eb67cd4923a840171833bb572a)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:11.431770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2726c-541c-4c84-84ad-1737b7da2e39 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:11.433330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33252db4-bc4a-4baa-a45c-fb44f4dbcb75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:11.434910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33252db4-bc4a-4baa-a45c-fb44f4dbcb75 service placement] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:11.445605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1214/2413] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:25:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:25:11.481203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c1b2726c-541c-4c84-84ad-1737b7da2e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c11f44eb67cd4923a840171833bb572a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:11.481929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c1b2726c-541c-4c84-84ad-1737b7da2e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c11f44eb67cd4923a840171833bb572a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:11.489593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c1b2726c-541c-4c84-84ad-1737b7da2e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c11f44eb67cd4923a840171833bb572a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:11.502351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c1b2726c-541c-4c84-84ad-1737b7da2e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c11f44eb67cd4923a840171833bb572a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:11.503015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-c1b2726c-541c-4c84-84ad-1737b7da2e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c11f44eb67cd4923a840171833bb572a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:11.515290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1200/2414] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:25:11 2022] DELETE /identity/v3/users/c11f44eb67cd4923a840171833bb572a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:25:11.684805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33252db4-bc4a-4baa-a45c-fb44f4dbcb75 service placement] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:11.696937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33252db4-bc4a-4baa-a45c-fb44f4dbcb75 service placement] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:11.755468 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:11.756360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:11.756649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:11.756930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9 admin admin] PATH_INFO: `/v3/projects/2b581a57e3e44e3481b1db445961afc5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:11.758092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b581a57e3e44e3481b1db445961afc5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:11.760001 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:11.804613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b581a57e3e44e3481b1db445961afc5', 'request_id': 'req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:11.810182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b581a57e3e44e3481b1db445961afc5', 'request_id': 'req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:11.816283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b581a57e3e44e3481b1db445961afc5', 'request_id': 'req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:11.817925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:11.829490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9 admin admin] The token cache is being invalidate because project 2b581a57e3e44e3481b1db445961afc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:25:11.829778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:11.830422 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-728d6d09-9c4b-4d87-825a-8fa1a6a85dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:11.831933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1215/2415] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:25:11 2022] DELETE /identity/v3/projects/2b581a57e3e44e3481b1db445961afc5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:25:11.837827 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624ae50-7d29-408b-8a2b-78cdf5fce4aa None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:11.838197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624ae50-7d29-408b-8a2b-78cdf5fce4aa None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:11.838549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624ae50-7d29-408b-8a2b-78cdf5fce4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:11.858804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-0624ae50-7d29-408b-8a2b-78cdf5fce4aa None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:11.860003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-0624ae50-7d29-408b-8a2b-78cdf5fce4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:11.917225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1201/2416] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:25:11.925544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fd00f-ee1b-428d-acda-527be37827b0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:11.925840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fd00f-ee1b-428d-acda-527be37827b0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:11.926123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fd00f-ee1b-428d-acda-527be37827b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:11.944127 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-450fd00f-ee1b-428d-acda-527be37827b0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:11.945371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-450fd00f-ee1b-428d-acda-527be37827b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:11.953313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1216/2417] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:25:11.957455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624ae50-7d29-408b-8a2b-78cdf5fce4aa None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:11.966964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624ae50-7d29-408b-8a2b-78cdf5fce4aa None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:11.971927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d302e5-ff59-4c0d-b5da-ce6fbfa843a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:11.972593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d302e5-ff59-4c0d-b5da-ce6fbfa843a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:11.972877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d302e5-ff59-4c0d-b5da-ce6fbfa843a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:11.973147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d302e5-ff59-4c0d-b5da-ce6fbfa843a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:11.973861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d302e5-ff59-4c0d-b5da-ce6fbfa843a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:25:11.974164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d302e5-ff59-4c0d-b5da-ce6fbfa843a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:11.975743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d302e5-ff59-4c0d-b5da-ce6fbfa843a0 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:11.987697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1202/2418] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:25:11.992116 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fd00f-ee1b-428d-acda-527be37827b0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:12.001322 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fd00f-ee1b-428d-acda-527be37827b0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:12.006000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34316ba-5d63-4eb9-8c14-b8b25873c964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:12.006650 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34316ba-5d63-4eb9-8c14-b8b25873c964 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:12.006944 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34316ba-5d63-4eb9-8c14-b8b25873c964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:12.007358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34316ba-5d63-4eb9-8c14-b8b25873c964 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:12.007991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34316ba-5d63-4eb9-8c14-b8b25873c964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:12.009952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34316ba-5d63-4eb9-8c14-b8b25873c964 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:12.035393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1217/2419] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:25:11 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:25:12.039412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d302e5-ff59-4c0d-b5da-ce6fbfa843a0 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:12.044783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:12.051446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d302e5-ff59-4c0d-b5da-ce6fbfa843a0 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:12.108730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4566c4e-a9c6-4484-ab8b-9af3f21cfa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:12.109396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4566c4e-a9c6-4484-ab8b-9af3f21cfa2a admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:12.109685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4566c4e-a9c6-4484-ab8b-9af3f21cfa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:12.109996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4566c4e-a9c6-4484-ab8b-9af3f21cfa2a admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:12.110687 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4566c4e-a9c6-4484-ab8b-9af3f21cfa2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:12.112623 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4566c4e-a9c6-4484-ab8b-9af3f21cfa2a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:12.131625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1203/2420] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:25:12 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:25:12.135641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34316ba-5d63-4eb9-8c14-b8b25873c964 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:12.146244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34316ba-5d63-4eb9-8c14-b8b25873c964 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:12.150847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70662106-5557-4817-800d-e0a3ff052e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:12.151499 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70662106-5557-4817-800d-e0a3ff052e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:12.151786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70662106-5557-4817-800d-e0a3ff052e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:12.152086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70662106-5557-4817-800d-e0a3ff052e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:12.152806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70662106-5557-4817-800d-e0a3ff052e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:12.154407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70662106-5557-4817-800d-e0a3ff052e03 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:12.165122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1218/2421] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:25:12.169210 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4566c4e-a9c6-4484-ab8b-9af3f21cfa2a admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:12.178908 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4566c4e-a9c6-4484-ab8b-9af3f21cfa2a admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:12.183483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20548835-7db5-476f-a679-41df9d5b0636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:12.184161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20548835-7db5-476f-a679-41df9d5b0636 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:12.184448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20548835-7db5-476f-a679-41df9d5b0636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:12.184723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20548835-7db5-476f-a679-41df9d5b0636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:12.185443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20548835-7db5-476f-a679-41df9d5b0636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:12.187009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20548835-7db5-476f-a679-41df9d5b0636 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:12.199855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1204/2422] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:25:12.200827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70662106-5557-4817-800d-e0a3ff052e03 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:12.211338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70662106-5557-4817-800d-e0a3ff052e03 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:12.216457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d947ef-4227-429b-8f81-6feebbbedeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:12.217392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d947ef-4227-429b-8f81-6feebbbedeea admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:12.217675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d947ef-4227-429b-8f81-6feebbbedeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:12.217952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d947ef-4227-429b-8f81-6feebbbedeea admin admin] PATH_INFO: `/v3/projects/9a7262e606264aba8be1c0008e93c0d2/users/de859eecbef7498289f132a047e6e5f8/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:12.234112 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d947ef-4227-429b-8f81-6feebbbedeea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7262e606264aba8be1c0008e93c0d2, user_id=de859eecbef7498289f132a047e6e5f8, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:12.236867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d947ef-4227-429b-8f81-6feebbbedeea admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:12.250153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1219/2423] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:25:12 2022] PUT /identity/v3/projects/9a7262e606264aba8be1c0008e93c0d2/users/de859eecbef7498289f132a047e6e5f8/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:25:12.255486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be014507-2a44-467d-b5ea-4ffe9c7bfea6 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:12.255896 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be014507-2a44-467d-b5ea-4ffe9c7bfea6 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:12.256143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be014507-2a44-467d-b5ea-4ffe9c7bfea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:12.274800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-be014507-2a44-467d-b5ea-4ffe9c7bfea6 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:12.276859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-be014507-2a44-467d-b5ea-4ffe9c7bfea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:12.338630 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1205/2424] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:25:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:25:12.348754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d947ef-4227-429b-8f81-6feebbbedeea admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:12.350554 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-58d947ef-4227-429b-8f81-6feebbbedeea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:12.408421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d947ef-4227-429b-8f81-6feebbbedeea admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:12.412262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c310e35-6493-49cd-85e1-cab5689a5fc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:12.412902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c310e35-6493-49cd-85e1-cab5689a5fc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:12.413183 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c310e35-6493-49cd-85e1-cab5689a5fc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:12.413457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c310e35-6493-49cd-85e1-cab5689a5fc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:12.422525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c310e35-6493-49cd-85e1-cab5689a5fc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:12.424266 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c310e35-6493-49cd-85e1-cab5689a5fc5 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:12.433527 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1220/2425] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:25:16.068607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c92c50-6c39-49b4-893b-f34f6acd9d18 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:16.069124 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c92c50-6c39-49b4-893b-f34f6acd9d18 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:16.069238 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c92c50-6c39-49b4-893b-f34f6acd9d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:16.096442 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-85c92c50-6c39-49b4-893b-f34f6acd9d18 None None] MFA Rules not processed for user `de859eecbef7498289f132a047e6e5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:16.097676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-85c92c50-6c39-49b4-893b-f34f6acd9d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:16.174331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1206/2426] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:25:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:25:16.190381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c310e35-6493-49cd-85e1-cab5689a5fc5 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:16.191899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7c310e35-6493-49cd-85e1-cab5689a5fc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:16.283821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c310e35-6493-49cd-85e1-cab5689a5fc5 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:16.298370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb6d1b-f6a0-4943-82d4-3e85d635a168 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:16.298370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb6d1b-f6a0-4943-82d4-3e85d635a168 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:16.298370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb6d1b-f6a0-4943-82d4-3e85d635a168 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:16.298370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb6d1b-f6a0-4943-82d4-3e85d635a168 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:16.316501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb6d1b-f6a0-4943-82d4-3e85d635a168 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:16.319847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb6d1b-f6a0-4943-82d4-3e85d635a168 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:16.336968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1221/2427] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:25:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:25:18.137027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b3a42-06f7-44d0-b6c6-2ae61d5e489e None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:18.137576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b3a42-06f7-44d0-b6c6-2ae61d5e489e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:18.137576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b3a42-06f7-44d0-b6c6-2ae61d5e489e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:18.160146 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-b27b3a42-06f7-44d0-b6c6-2ae61d5e489e None None] MFA Rules not processed for user `de859eecbef7498289f132a047e6e5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:18.161396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b27b3a42-06f7-44d0-b6c6-2ae61d5e489e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:18.169070 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1207/2428] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:25:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:25:18.548030 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb6d1b-f6a0-4943-82d4-3e85d635a168 service nova] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:18.549388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-4ccb6d1b-f6a0-4943-82d4-3e85d635a168 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:18.564519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb6d1b-f6a0-4943-82d4-3e85d635a168 service nova] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:18.568552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec3ca8-d346-4581-88f1-515d7ac984ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:18.569209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec3ca8-d346-4581-88f1-515d7ac984ba service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:18.569495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec3ca8-d346-4581-88f1-515d7ac984ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:18.569775 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec3ca8-d346-4581-88f1-515d7ac984ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:18.571578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-bbec3ca8-d346-4581-88f1-515d7ac984ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:18.582385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec3ca8-d346-4581-88f1-515d7ac984ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:18.583841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec3ca8-d346-4581-88f1-515d7ac984ba service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:18.593385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1222/2429] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:25:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:25:19.082722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b3a42-06f7-44d0-b6c6-2ae61d5e489e None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:19.095967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:19.106773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b3a42-06f7-44d0-b6c6-2ae61d5e489e None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:19.115713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b992f-0ea8-4169-b9d5-5cf446298f75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:19.116422 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b992f-0ea8-4169-b9d5-5cf446298f75 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:19.116714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b992f-0ea8-4169-b9d5-5cf446298f75 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:19.116993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b992f-0ea8-4169-b9d5-5cf446298f75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:19.118826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f94b992f-0ea8-4169-b9d5-5cf446298f75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:19.138001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b992f-0ea8-4169-b9d5-5cf446298f75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:19.140103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b992f-0ea8-4169-b9d5-5cf446298f75 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:19.155871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1208/2430] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:25:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:25:22.148371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec3ca8-d346-4581-88f1-515d7ac984ba service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:22.161077 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:22.171745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec3ca8-d346-4581-88f1-515d7ac984ba service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:22.181159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc5a59-9e2e-4b23-8721-e3dfdadd5eee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:22.182057 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc5a59-9e2e-4b23-8721-e3dfdadd5eee service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:22.182353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc5a59-9e2e-4b23-8721-e3dfdadd5eee service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:22.182671 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc5a59-9e2e-4b23-8721-e3dfdadd5eee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:22.184518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-50fc5a59-9e2e-4b23-8721-e3dfdadd5eee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:22.198883 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc5a59-9e2e-4b23-8721-e3dfdadd5eee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:22.200623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc5a59-9e2e-4b23-8721-e3dfdadd5eee service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:22.213241 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1223/2431] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:25:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:25:27.645602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aace0-e4b1-4b8e-8a20-9ffb5d240bfe None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:27.646033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aace0-e4b1-4b8e-8a20-9ffb5d240bfe None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:27.646163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aace0-e4b1-4b8e-8a20-9ffb5d240bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:27.663477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-795aace0-e4b1-4b8e-8a20-9ffb5d240bfe None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:27.664722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-795aace0-e4b1-4b8e-8a20-9ffb5d240bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:27.718632 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1209/2432] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jun 17 07:25:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:25:27.765047 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc5a59-9e2e-4b23-8721-e3dfdadd5eee service nova] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:27.766452 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-50fc5a59-9e2e-4b23-8721-e3dfdadd5eee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:27.848174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc5a59-9e2e-4b23-8721-e3dfdadd5eee service nova] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:27.853138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d5889-6d48-48ca-98bb-16111e6d227b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:27.853784 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d5889-6d48-48ca-98bb-16111e6d227b service swift] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:27.854071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d5889-6d48-48ca-98bb-16111e6d227b service swift] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:27.854352 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d5889-6d48-48ca-98bb-16111e6d227b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:27.862871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d5889-6d48-48ca-98bb-16111e6d227b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:27.864502 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d5889-6d48-48ca-98bb-16111e6d227b service swift] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:27.872456 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1224/2433] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:25:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:25:29.278126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aace0-e4b1-4b8e-8a20-9ffb5d240bfe None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:29.293472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aace0-e4b1-4b8e-8a20-9ffb5d240bfe None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:29.300659 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2a3c2-b92e-4038-b8c3-3092f66965e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:29.301319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2a3c2-b92e-4038-b8c3-3092f66965e3 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:29.301606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2a3c2-b92e-4038-b8c3-3092f66965e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:29.301907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2a3c2-b92e-4038-b8c3-3092f66965e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:29.303679 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-33b2a3c2-b92e-4038-b8c3-3092f66965e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:29.377024 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2a3c2-b92e-4038-b8c3-3092f66965e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:29.378563 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2a3c2-b92e-4038-b8c3-3092f66965e3 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:29.392250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1210/2434] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:25:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:25:34.916374 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d5889-6d48-48ca-98bb-16111e6d227b service swift] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:34.917818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ba5d5889-6d48-48ca-98bb-16111e6d227b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:34.932801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d5889-6d48-48ca-98bb-16111e6d227b service swift] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:34.937397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:34.938270 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:34.938560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:34.938869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] PATH_INFO: `/v3/users/a21116ea64234269bba8b0369fb19b59` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:34.940059 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a21116ea64234269bba8b0369fb19b59)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:34.942021 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:34.987123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a21116ea64234269bba8b0369fb19b59'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:34.994551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a21116ea64234269bba8b0369fb19b59'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:34.995142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a21116ea64234269bba8b0369fb19b59'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.006749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a21116ea64234269bba8b0369fb19b59'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.007317 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a21116ea64234269bba8b0369fb19b59'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.019837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1225/2435] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:25:34 2022] DELETE /identity/v3/users/a21116ea64234269bba8b0369fb19b59 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:25:35.195461 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2a3c2-b92e-4038-b8c3-3092f66965e3 service nova] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:35.208568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2a3c2-b92e-4038-b8c3-3092f66965e3 service nova] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:35.265065 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:35.265858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:35.266142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:35.266420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] PATH_INFO: `/v3/projects/52a029f97a9e4142802a9786ae2f6d58` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:35.267604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52a029f97a9e4142802a9786ae2f6d58)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:35.269579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:35.308000 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:35.317713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52a029f97a9e4142802a9786ae2f6d58', 'request_id': 'req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.324989 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52a029f97a9e4142802a9786ae2f6d58', 'request_id': 'req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.331577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52a029f97a9e4142802a9786ae2f6d58', 'request_id': 'req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.341604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] The token cache is being invalidate because project 52a029f97a9e4142802a9786ae2f6d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:25:35.341892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.343059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.344620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1211/2436] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:25:35 2022] DELETE /identity/v3/projects/52a029f97a9e4142802a9786ae2f6d58 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:25:35.348588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:35.349970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:35.362724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81e254-72f7-4e45-8cff-4c746b8620f6 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:35.401455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:35.421193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85db92f-486d-4033-9acc-55096944afd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:35.422057 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85db92f-486d-4033-9acc-55096944afd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:35.422340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85db92f-486d-4033-9acc-55096944afd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:35.422614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85db92f-486d-4033-9acc-55096944afd4 admin admin] PATH_INFO: `/v3/users/bd32f57db67345c8b524b624cca5c901` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:35.423814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85db92f-486d-4033-9acc-55096944afd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd32f57db67345c8b524b624cca5c901)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:35.425779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85db92f-486d-4033-9acc-55096944afd4 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:35.471737 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e85db92f-486d-4033-9acc-55096944afd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd32f57db67345c8b524b624cca5c901'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.478067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e85db92f-486d-4033-9acc-55096944afd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd32f57db67345c8b524b624cca5c901'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.478839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e85db92f-486d-4033-9acc-55096944afd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd32f57db67345c8b524b624cca5c901'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.490356 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e85db92f-486d-4033-9acc-55096944afd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd32f57db67345c8b524b624cca5c901'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.490979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-e85db92f-486d-4033-9acc-55096944afd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd32f57db67345c8b524b624cca5c901'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:35.503510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1226/2437] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:25:35 2022] DELETE /identity/v3/users/bd32f57db67345c8b524b624cca5c901 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:25:35.968598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:35.970070 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:35.983146 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b111a-c913-4dcd-be50-f29f14f6f8d9 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:36.048885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b5732-5915-4bd3-96a3-4a9d6ba94748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:36.049739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b5732-5915-4bd3-96a3-4a9d6ba94748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:36.050030 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b5732-5915-4bd3-96a3-4a9d6ba94748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:36.050319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b5732-5915-4bd3-96a3-4a9d6ba94748 admin admin] PATH_INFO: `/v3/projects/1cdfcd3d2aa84226bc01f6400dd9754c` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:36.051468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b5732-5915-4bd3-96a3-4a9d6ba94748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cdfcd3d2aa84226bc01f6400dd9754c)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:36.053446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b5732-5915-4bd3-96a3-4a9d6ba94748 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:36.097430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e77b5732-5915-4bd3-96a3-4a9d6ba94748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cdfcd3d2aa84226bc01f6400dd9754c', 'request_id': 'req-e77b5732-5915-4bd3-96a3-4a9d6ba94748'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:36.104477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e77b5732-5915-4bd3-96a3-4a9d6ba94748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cdfcd3d2aa84226bc01f6400dd9754c', 'request_id': 'req-e77b5732-5915-4bd3-96a3-4a9d6ba94748'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:36.111002 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e77b5732-5915-4bd3-96a3-4a9d6ba94748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cdfcd3d2aa84226bc01f6400dd9754c', 'request_id': 'req-e77b5732-5915-4bd3-96a3-4a9d6ba94748'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:36.120967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e77b5732-5915-4bd3-96a3-4a9d6ba94748 admin admin] The token cache is being invalidate because project 1cdfcd3d2aa84226bc01f6400dd9754c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:25:36.121248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e77b5732-5915-4bd3-96a3-4a9d6ba94748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:36.121860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e77b5732-5915-4bd3-96a3-4a9d6ba94748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:25:36.124114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1212/2438] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:25:35 2022] DELETE /identity/v3/projects/1cdfcd3d2aa84226bc01f6400dd9754c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:25:36.129044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e4a09-5920-49ec-8120-7be1b62c068c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:36.129515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e4a09-5920-49ec-8120-7be1b62c068c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:36.129628 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e4a09-5920-49ec-8120-7be1b62c068c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:36.147239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c23e4a09-5920-49ec-8120-7be1b62c068c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:36.148463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c23e4a09-5920-49ec-8120-7be1b62c068c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:36.205473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1227/2439] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:25:36.213151 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800d550-dc87-4326-bab2-2d03f63e316c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:36.213588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800d550-dc87-4326-bab2-2d03f63e316c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:36.213711 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800d550-dc87-4326-bab2-2d03f63e316c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:36.233351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-8800d550-dc87-4326-bab2-2d03f63e316c None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:36.234563 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-8800d550-dc87-4326-bab2-2d03f63e316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:36.242321 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1213/2440] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:25:36.251875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e4a09-5920-49ec-8120-7be1b62c068c None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:36.262845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e4a09-5920-49ec-8120-7be1b62c068c None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:36.267570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666742a-f027-4fb5-861b-599da1b7d969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:36.268254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666742a-f027-4fb5-861b-599da1b7d969 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:36.268565 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666742a-f027-4fb5-861b-599da1b7d969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:36.268838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666742a-f027-4fb5-861b-599da1b7d969 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:36.269560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666742a-f027-4fb5-861b-599da1b7d969 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:25:36.269868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666742a-f027-4fb5-861b-599da1b7d969 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:36.271459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666742a-f027-4fb5-861b-599da1b7d969 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:36.283810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1228/2441] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:25:36.288439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800d550-dc87-4326-bab2-2d03f63e316c None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:36.298975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800d550-dc87-4326-bab2-2d03f63e316c None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:36.303604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4fd68-4599-4797-af67-f4d951235b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:36.304283 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4fd68-4599-4797-af67-f4d951235b77 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:36.304571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4fd68-4599-4797-af67-f4d951235b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:36.304849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4fd68-4599-4797-af67-f4d951235b77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:36.305535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4fd68-4599-4797-af67-f4d951235b77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:36.307432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4fd68-4599-4797-af67-f4d951235b77 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:36.332072 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1214/2442] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:25:36 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:25:36.336114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666742a-f027-4fb5-861b-599da1b7d969 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:36.346639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666742a-f027-4fb5-861b-599da1b7d969 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:36.389467 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:36.405241 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4d8be-4af6-4d68-9be2-281a27004f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:36.405949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4d8be-4af6-4d68-9be2-281a27004f07 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:36.406240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4d8be-4af6-4d68-9be2-281a27004f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:36.406525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4d8be-4af6-4d68-9be2-281a27004f07 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:36.407246 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4d8be-4af6-4d68-9be2-281a27004f07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:36.409230 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4d8be-4af6-4d68-9be2-281a27004f07 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:36.428416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1229/2443] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:25:36 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:25:36.432487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4fd68-4599-4797-af67-f4d951235b77 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:36.442680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4fd68-4599-4797-af67-f4d951235b77 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:36.447291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4961adb2-2ce5-4211-a3d0-799afc8e93e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:36.447956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961adb2-2ce5-4211-a3d0-799afc8e93e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:36.448282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961adb2-2ce5-4211-a3d0-799afc8e93e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:36.448561 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961adb2-2ce5-4211-a3d0-799afc8e93e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:36.449292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4961adb2-2ce5-4211-a3d0-799afc8e93e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:36.451243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4961adb2-2ce5-4211-a3d0-799afc8e93e5 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:36.462026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1215/2444] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:25:36.465970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4d8be-4af6-4d68-9be2-281a27004f07 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:36.476608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4d8be-4af6-4d68-9be2-281a27004f07 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:36.481385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa79b84-b616-450d-afe8-9febda3139f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:36.482052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa79b84-b616-450d-afe8-9febda3139f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:36.482341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa79b84-b616-450d-afe8-9febda3139f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:36.482628 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa79b84-b616-450d-afe8-9febda3139f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:36.483921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa79b84-b616-450d-afe8-9febda3139f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:36.485570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa79b84-b616-450d-afe8-9febda3139f0 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:36.496290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1230/2445] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:25:36.500614 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4961adb2-2ce5-4211-a3d0-799afc8e93e5 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:36.510755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4961adb2-2ce5-4211-a3d0-799afc8e93e5 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:36.515857 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d4047-d05a-4a8f-8072-5c4c9d0182aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:36.516774 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d4047-d05a-4a8f-8072-5c4c9d0182aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:36.517139 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d4047-d05a-4a8f-8072-5c4c9d0182aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:36.517421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d4047-d05a-4a8f-8072-5c4c9d0182aa admin admin] PATH_INFO: `/v3/projects/d2b7ae6a418640629020f39ac455d075/users/6a113fe9b988458d8e8898c5c9a8d02d/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:36.529362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:36.533781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d4047-d05a-4a8f-8072-5c4c9d0182aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b7ae6a418640629020f39ac455d075, user_id=6a113fe9b988458d8e8898c5c9a8d02d, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:36.537557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d4047-d05a-4a8f-8072-5c4c9d0182aa admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:36.551872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1216/2446] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:25:36 2022] PUT /identity/v3/projects/d2b7ae6a418640629020f39ac455d075/users/6a113fe9b988458d8e8898c5c9a8d02d/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:25:36.557176 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728a675-70a2-4124-8c6e-07252603e207 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:36.557473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728a675-70a2-4124-8c6e-07252603e207 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:36.557758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728a675-70a2-4124-8c6e-07252603e207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:36.584912 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-7728a675-70a2-4124-8c6e-07252603e207 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:36.586157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7728a675-70a2-4124-8c6e-07252603e207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:36.644119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1231/2447] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:25:36.653514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d4047-d05a-4a8f-8072-5c4c9d0182aa admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:36.655765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-827d4047-d05a-4a8f-8072-5c4c9d0182aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:36.714681 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d4047-d05a-4a8f-8072-5c4c9d0182aa admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:36.721423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011351c-10a6-4659-b5a8-20e7578aac32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:36.723247 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011351c-10a6-4659-b5a8-20e7578aac32 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:36.723544 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011351c-10a6-4659-b5a8-20e7578aac32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:36.724056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011351c-10a6-4659-b5a8-20e7578aac32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:36.735084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011351c-10a6-4659-b5a8-20e7578aac32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:36.736584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011351c-10a6-4659-b5a8-20e7578aac32 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:36.746755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1217/2448] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:25:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:25:41.014957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c20e1-709f-4099-8e2a-e5fb4a90182e None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:41.015463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c20e1-709f-4099-8e2a-e5fb4a90182e None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:41.015582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c20e1-709f-4099-8e2a-e5fb4a90182e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:41.049466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-031c20e1-709f-4099-8e2a-e5fb4a90182e None None] MFA Rules not processed for user `6a113fe9b988458d8e8898c5c9a8d02d`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:41.050837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-031c20e1-709f-4099-8e2a-e5fb4a90182e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:41.153783 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1232/2449] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:25:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:25:41.158635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011351c-10a6-4659-b5a8-20e7578aac32 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:41.169762 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011351c-10a6-4659-b5a8-20e7578aac32 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:41.175387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2477cef-be58-4cb0-ba5a-7658399c5d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:41.176284 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2477cef-be58-4cb0-ba5a-7658399c5d08 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:41.176768 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2477cef-be58-4cb0-ba5a-7658399c5d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:41.177158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2477cef-be58-4cb0-ba5a-7658399c5d08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:41.178150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2477cef-be58-4cb0-ba5a-7658399c5d08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:41.180190 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2477cef-be58-4cb0-ba5a-7658399c5d08 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:41.211604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1218/2450] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:25:41 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:25:41.212981 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c20e1-709f-4099-8e2a-e5fb4a90182e None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:41.226227 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c20e1-709f-4099-8e2a-e5fb4a90182e None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:41.282399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:41.292303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9556b-0bd2-4acc-8d8c-2e6ac5b5a9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:41.293114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9556b-0bd2-4acc-8d8c-2e6ac5b5a9b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:41.293556 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9556b-0bd2-4acc-8d8c-2e6ac5b5a9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:41.293943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9556b-0bd2-4acc-8d8c-2e6ac5b5a9b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:41.295175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9556b-0bd2-4acc-8d8c-2e6ac5b5a9b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:41.297262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9556b-0bd2-4acc-8d8c-2e6ac5b5a9b0 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:41.321488 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1233/2451] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:25:41 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:25:41.322681 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2477cef-be58-4cb0-ba5a-7658399c5d08 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:41.334396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2477cef-be58-4cb0-ba5a-7658399c5d08 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:41.339203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32104b4f-705d-4676-bbc7-b2fa5e0f94d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:41.339911 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32104b4f-705d-4676-bbc7-b2fa5e0f94d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:41.340237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32104b4f-705d-4676-bbc7-b2fa5e0f94d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:41.340521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32104b4f-705d-4676-bbc7-b2fa5e0f94d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:41.341403 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32104b4f-705d-4676-bbc7-b2fa5e0f94d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:41.343606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32104b4f-705d-4676-bbc7-b2fa5e0f94d9 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:41.355414 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1219/2452] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:25:41.359789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9556b-0bd2-4acc-8d8c-2e6ac5b5a9b0 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:41.370555 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9556b-0bd2-4acc-8d8c-2e6ac5b5a9b0 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:41.376039 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560a530-2b4f-4e2e-8714-9f12c546ea75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:41.376970 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560a530-2b4f-4e2e-8714-9f12c546ea75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:41.377263 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560a530-2b4f-4e2e-8714-9f12c546ea75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:41.377576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560a530-2b4f-4e2e-8714-9f12c546ea75 admin admin] PATH_INFO: `/v3/projects/497160218e904aff92a91866c5ba3710/users/21d37c60639641d4a2e39141346d272e/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:41.394995 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560a530-2b4f-4e2e-8714-9f12c546ea75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=497160218e904aff92a91866c5ba3710, user_id=21d37c60639641d4a2e39141346d272e, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:41.397802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560a530-2b4f-4e2e-8714-9f12c546ea75 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:41.412511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1234/2453] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:25:41 2022] PUT /identity/v3/projects/497160218e904aff92a91866c5ba3710/users/21d37c60639641d4a2e39141346d272e/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:25:41.416646 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32104b4f-705d-4676-bbc7-b2fa5e0f94d9 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:41.428049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32104b4f-705d-4676-bbc7-b2fa5e0f94d9 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:41.490988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11180090-1ef6-4bd3-af1a-ef0e2f56de4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:41.491745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11180090-1ef6-4bd3-af1a-ef0e2f56de4d admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:41.492192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11180090-1ef6-4bd3-af1a-ef0e2f56de4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:41.492489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11180090-1ef6-4bd3-af1a-ef0e2f56de4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:41.493231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11180090-1ef6-4bd3-af1a-ef0e2f56de4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:41.495792 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11180090-1ef6-4bd3-af1a-ef0e2f56de4d admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:41.507675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1220/2454] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:25:41.511788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560a530-2b4f-4e2e-8714-9f12c546ea75 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:41.523272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560a530-2b4f-4e2e-8714-9f12c546ea75 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:41.537881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a928dd-52d9-4967-b5d9-aaf25691981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:41.539362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a928dd-52d9-4967-b5d9-aaf25691981e admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:41.539856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a928dd-52d9-4967-b5d9-aaf25691981e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:41.540265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a928dd-52d9-4967-b5d9-aaf25691981e admin admin] PATH_INFO: `/v3/projects/497160218e904aff92a91866c5ba3710/users/21d37c60639641d4a2e39141346d272e/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:41.543406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a928dd-52d9-4967-b5d9-aaf25691981e admin admin] RBAC: Authorizing `identity:create_grant(project_id=497160218e904aff92a91866c5ba3710, user_id=21d37c60639641d4a2e39141346d272e, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:41.549119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a928dd-52d9-4967-b5d9-aaf25691981e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:41.560737 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1235/2455] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:25:41 2022] PUT /identity/v3/projects/497160218e904aff92a91866c5ba3710/users/21d37c60639641d4a2e39141346d272e/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:25:46.641978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be8699-2fdf-40da-9c81-52454f60fe33 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:46.642505 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be8699-2fdf-40da-9c81-52454f60fe33 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:46.642611 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be8699-2fdf-40da-9c81-52454f60fe33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:46.672763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-72be8699-2fdf-40da-9c81-52454f60fe33 None None] MFA Rules not processed for user `21d37c60639641d4a2e39141346d272e`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:46.674029 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-72be8699-2fdf-40da-9c81-52454f60fe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:46.723898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:46.754874 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1221/2456] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:25:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:25:46.825221 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a928dd-52d9-4967-b5d9-aaf25691981e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:46.829364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-21a928dd-52d9-4967-b5d9-aaf25691981e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:46.920977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a928dd-52d9-4967-b5d9-aaf25691981e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:46.930165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457150e8-aca6-47a9-89d5-d187e786e33b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:46.930165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457150e8-aca6-47a9-89d5-d187e786e33b service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:46.930512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457150e8-aca6-47a9-89d5-d187e786e33b service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:46.930742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457150e8-aca6-47a9-89d5-d187e786e33b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:47.000028 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457150e8-aca6-47a9-89d5-d187e786e33b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:47.001809 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457150e8-aca6-47a9-89d5-d187e786e33b service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:47.012999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1236/2457] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:25:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:25:50.119077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be8699-2fdf-40da-9c81-52454f60fe33 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:50.120614 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-72be8699-2fdf-40da-9c81-52454f60fe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:50.145641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be8699-2fdf-40da-9c81-52454f60fe33 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:50.151944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3d42b-f19a-41ce-a21d-5106463d79f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:50.152993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3d42b-f19a-41ce-a21d-5106463d79f4 service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:50.153487 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3d42b-f19a-41ce-a21d-5106463d79f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:50.154388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3d42b-f19a-41ce-a21d-5106463d79f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:50.157056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a8f3d42b-f19a-41ce-a21d-5106463d79f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:50.175591 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3d42b-f19a-41ce-a21d-5106463d79f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:50.177641 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3d42b-f19a-41ce-a21d-5106463d79f4 service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:50.218538 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1222/2458] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:25:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:25:51.228843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b43436-da3f-42e7-ad7e-93629a10bea6 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:51.229371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b43436-da3f-42e7-ad7e-93629a10bea6 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:51.229484 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b43436-da3f-42e7-ad7e-93629a10bea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:51.248139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-35b43436-da3f-42e7-ad7e-93629a10bea6 None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:25:51.249983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-35b43436-da3f-42e7-ad7e-93629a10bea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:51.310180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:25:51.336113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1237/2459] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:25:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:25:51.343106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3d42b-f19a-41ce-a21d-5106463d79f4 service nova] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:25:51.344695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a8f3d42b-f19a-41ce-a21d-5106463d79f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:25:51.438723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3d42b-f19a-41ce-a21d-5106463d79f4 service nova] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:25:51.443739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3375aa4-76c7-4804-b4d3-24f8bfdcce7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:25:51.444694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3375aa4-76c7-4804-b4d3-24f8bfdcce7d service swift] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:25:51.445096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3375aa4-76c7-4804-b4d3-24f8bfdcce7d service swift] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:25:51.445937 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3375aa4-76c7-4804-b4d3-24f8bfdcce7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:25:51.457898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3375aa4-76c7-4804-b4d3-24f8bfdcce7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:25:51.459516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3375aa4-76c7-4804-b4d3-24f8bfdcce7d service swift] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:25:51.470411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1223/2460] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:25:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:26:01.321209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:01.465687 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=46 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:04.713457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb959722-d7dc-4e19-bf4b-91f9c73f8eb2 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:04.714279 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb959722-d7dc-4e19-bf4b-91f9c73f8eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:04.714789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb959722-d7dc-4e19-bf4b-91f9c73f8eb2 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:04.716258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1238/2461] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:26:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:26:04.724514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc91ab6-538a-42d0-850e-3ee96623716f None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:04.725540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc91ab6-538a-42d0-850e-3ee96623716f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:04.725892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc91ab6-538a-42d0-850e-3ee96623716f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:04.746747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ddc91ab6-538a-42d0-850e-3ee96623716f None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:04.747954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ddc91ab6-538a-42d0-850e-3ee96623716f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:04.757197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1224/2462] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:26:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:04.769164 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb959722-d7dc-4e19-bf4b-91f9c73f8eb2 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:04.770898 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-bb959722-d7dc-4e19-bf4b-91f9c73f8eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:04.866896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb959722-d7dc-4e19-bf4b-91f9c73f8eb2 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:04.870542 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808b98a-9337-49fb-a4c7-d4c680ad6493 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48586ff1269f49a3ae3d9f3e40c1b9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:04.871190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808b98a-9337-49fb-a4c7-d4c680ad6493 service glance] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:04.871475 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808b98a-9337-49fb-a4c7-d4c680ad6493 service glance] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:04.871752 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808b98a-9337-49fb-a4c7-d4c680ad6493 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:04.882290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808b98a-9337-49fb-a4c7-d4c680ad6493 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:04.883717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808b98a-9337-49fb-a4c7-d4c680ad6493 service glance] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:04.894865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1239/2463] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:04.961689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe99b9e-51a2-4783-95f5-93e0a11fd17d None None] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:04.962007 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe99b9e-51a2-4783-95f5-93e0a11fd17d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:04.962290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe99b9e-51a2-4783-95f5-93e0a11fd17d None None] PATH_INFO: `/` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:04.963674 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1225/2464] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:26:04.969612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116b8ad-2067-46dd-8e02-d0403c14b51c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:04.969910 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116b8ad-2067-46dd-8e02-d0403c14b51c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:04.970193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116b8ad-2067-46dd-8e02-d0403c14b51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:04.988020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-5116b8ad-2067-46dd-8e02-d0403c14b51c None None] MFA Rules not processed for user `7e0797679c07454abc867cd7b30c8b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:04.989271 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5116b8ad-2067-46dd-8e02-d0403c14b51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:05.040199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1240/2465] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:26:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:26:05.051938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe99b9e-51a2-4783-95f5-93e0a11fd17d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:05.053666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-dbe99b9e-51a2-4783-95f5-93e0a11fd17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:05.067585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe99b9e-51a2-4783-95f5-93e0a11fd17d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:05.071520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef5efd-b256-4226-87a9-f4813abe0eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:05.072196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef5efd-b256-4226-87a9-f4813abe0eef service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:05.072482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef5efd-b256-4226-87a9-f4813abe0eef service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:05.072759 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef5efd-b256-4226-87a9-f4813abe0eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:05.081736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef5efd-b256-4226-87a9-f4813abe0eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:05.083207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef5efd-b256-4226-87a9-f4813abe0eef service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:05.092575 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1226/2466] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:05.193839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c0143-f16c-49eb-bca3-b612c7d5b426 None None] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:05.194631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c0143-f16c-49eb-bca3-b612c7d5b426 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:05.194921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c0143-f16c-49eb-bca3-b612c7d5b426 None None] PATH_INFO: `/` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:05.196528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1241/2467] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jun 17 07:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 07:26:05.200480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9a771-0611-4abf-b428-d1b8595c94c1 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:05.200765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9a771-0611-4abf-b428-d1b8595c94c1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:05.201155 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9a771-0611-4abf-b428-d1b8595c94c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:05.205734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cbdd0-5d1f-431b-b120-0e14786574d1 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:05.206026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cbdd0-5d1f-431b-b120-0e14786574d1 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:05.206310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cbdd0-5d1f-431b-b120-0e14786574d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:05.229950 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-6fa9a771-0611-4abf-b428-d1b8595c94c1 None None] MFA Rules not processed for user `6a113fe9b988458d8e8898c5c9a8d02d`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:05.231318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6fa9a771-0611-4abf-b428-d1b8595c94c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:05.232292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-da8cbdd0-5d1f-431b-b120-0e14786574d1 None None] MFA Rules not processed for user `8d3baa16824f40f3a4c783e210b262f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:05.234945 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-da8cbdd0-5d1f-431b-b120-0e14786574d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:05.241138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1227/2468] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:05.247501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1242/2469] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jun 17 07:26:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:05.259531 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9a771-0611-4abf-b428-d1b8595c94c1 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:05.270058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9a771-0611-4abf-b428-d1b8595c94c1 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:05.273836 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9c2e2-3745-4121-94c8-7a4d9f4e0351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:05.274517 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9c2e2-3745-4121-94c8-7a4d9f4e0351 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:05.274785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9c2e2-3745-4121-94c8-7a4d9f4e0351 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:05.275046 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9c2e2-3745-4121-94c8-7a4d9f4e0351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:05.288480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9c2e2-3745-4121-94c8-7a4d9f4e0351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:05.290522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9c2e2-3745-4121-94c8-7a4d9f4e0351 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:05.304231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1228/2470] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:07.785863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31778fa8-7d4e-433f-9401-ff1d3907564f None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:07.786416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31778fa8-7d4e-433f-9401-ff1d3907564f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:07.786416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31778fa8-7d4e-433f-9401-ff1d3907564f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:07.805840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-31778fa8-7d4e-433f-9401-ff1d3907564f None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:07.807485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-31778fa8-7d4e-433f-9401-ff1d3907564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:07.829425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9c2e2-3745-4121-94c8-7a4d9f4e0351 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:07.848015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1243/2471] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jun 17 07:26:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:07.855664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9c2e2-3745-4121-94c8-7a4d9f4e0351 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:07.864879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89280f-d3ad-4eea-baac-c3cf785d2e44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:07.866595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89280f-d3ad-4eea-baac-c3cf785d2e44 service swift] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:07.867314 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89280f-d3ad-4eea-baac-c3cf785d2e44 service swift] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:07.867732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89280f-d3ad-4eea-baac-c3cf785d2e44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:07.888197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89280f-d3ad-4eea-baac-c3cf785d2e44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:07.891568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89280f-d3ad-4eea-baac-c3cf785d2e44 service swift] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:07.902441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1229/2472] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:26:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:26:17.800131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=64 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:17.898468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=24 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:23.842596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae42317-944c-4ddc-b0d5-898e971e830c None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:23.843143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae42317-944c-4ddc-b0d5-898e971e830c None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:23.843253 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae42317-944c-4ddc-b0d5-898e971e830c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:23.866073 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-0ae42317-944c-4ddc-b0d5-898e971e830c None None] MFA Rules not processed for user `dc98526f14e84279b002c43ef44cabed`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:23.867311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-0ae42317-944c-4ddc-b0d5-898e971e830c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:23.876867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1244/2473] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jun 17 07:26:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:26:23.897239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89280f-d3ad-4eea-baac-c3cf785d2e44 service swift] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:23.911204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89280f-d3ad-4eea-baac-c3cf785d2e44 service swift] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:23.920086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b94c63-89f1-46bd-b350-f6a778ae5f26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '004e5909f9834cfea890063bb0083d7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:23.922888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b94c63-89f1-46bd-b350-f6a778ae5f26 service swift] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:23.923705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b94c63-89f1-46bd-b350-f6a778ae5f26 service swift] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:23.924241 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b94c63-89f1-46bd-b350-f6a778ae5f26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:23.937808 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b94c63-89f1-46bd-b350-f6a778ae5f26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:23.939883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b94c63-89f1-46bd-b350-f6a778ae5f26 service swift] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:23.964154 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1230/2474] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jun 17 07:26:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 07:26:28.138952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7cee9-a2e0-44e2-b963-0ea03459d94f None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:28.139562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7cee9-a2e0-44e2-b963-0ea03459d94f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:28.139562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7cee9-a2e0-44e2-b963-0ea03459d94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:28.160241 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6da7cee9-a2e0-44e2-b963-0ea03459d94f None None] MFA Rules not processed for user `de859eecbef7498289f132a047e6e5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:28.161485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6da7cee9-a2e0-44e2-b963-0ea03459d94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:28.235310 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1245/2475] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:28.571991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b94c63-89f1-46bd-b350-f6a778ae5f26 service swift] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:28.585320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b94c63-89f1-46bd-b350-f6a778ae5f26 service swift] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:28.589256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1170282-564c-4f1b-8684-d7abf2f6096c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:28.589996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1170282-564c-4f1b-8684-d7abf2f6096c service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:28.590345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1170282-564c-4f1b-8684-d7abf2f6096c service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:28.590690 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1170282-564c-4f1b-8684-d7abf2f6096c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:28.592640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a1170282-564c-4f1b-8684-d7abf2f6096c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:28.605339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1170282-564c-4f1b-8684-d7abf2f6096c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:28.606055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1170282-564c-4f1b-8684-d7abf2f6096c service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:28.617699 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1231/2476] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:30.066295 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7cee9-a2e0-44e2-b963-0ea03459d94f None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:30.077625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7cee9-a2e0-44e2-b963-0ea03459d94f None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:30.081612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c9879-06ef-4c25-a6e8-32018deb54e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:30.082265 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c9879-06ef-4c25-a6e8-32018deb54e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:30.082742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c9879-06ef-4c25-a6e8-32018deb54e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:30.083028 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c9879-06ef-4c25-a6e8-32018deb54e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:30.085023 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-2a8c9879-06ef-4c25-a6e8-32018deb54e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:30.179613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c9879-06ef-4c25-a6e8-32018deb54e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:30.184428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c9879-06ef-4c25-a6e8-32018deb54e5 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:30.205744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1246/2477] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:33.395916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1170282-564c-4f1b-8684-d7abf2f6096c service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:33.405011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1170282-564c-4f1b-8684-d7abf2f6096c service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:33.410009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a755ba-0813-4786-ad72-95e35400387f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:33.410526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a755ba-0813-4786-ad72-95e35400387f service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:33.410808 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a755ba-0813-4786-ad72-95e35400387f service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:33.411083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a755ba-0813-4786-ad72-95e35400387f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:33.423437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a755ba-0813-4786-ad72-95e35400387f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:33.425147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a755ba-0813-4786-ad72-95e35400387f service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:33.437033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1232/2478] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:34.409777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c9879-06ef-4c25-a6e8-32018deb54e5 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:34.411492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-2a8c9879-06ef-4c25-a6e8-32018deb54e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:34.429229 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c9879-06ef-4c25-a6e8-32018deb54e5 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:34.437663 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6408e91-41e1-415e-9fc1-338372579d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:34.438940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6408e91-41e1-415e-9fc1-338372579d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:34.439758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6408e91-41e1-415e-9fc1-338372579d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:34.439758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6408e91-41e1-415e-9fc1-338372579d70 admin admin] PATH_INFO: `/v3/users/b3ca63f8f8484ddf9d7c72749ba7f482` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:34.442433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6408e91-41e1-415e-9fc1-338372579d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3ca63f8f8484ddf9d7c72749ba7f482)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:34.446522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6408e91-41e1-415e-9fc1-338372579d70 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:34.514512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d6408e91-41e1-415e-9fc1-338372579d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3ca63f8f8484ddf9d7c72749ba7f482'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:34.530281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d6408e91-41e1-415e-9fc1-338372579d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3ca63f8f8484ddf9d7c72749ba7f482'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:34.531072 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d6408e91-41e1-415e-9fc1-338372579d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3ca63f8f8484ddf9d7c72749ba7f482'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:34.545369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d6408e91-41e1-415e-9fc1-338372579d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3ca63f8f8484ddf9d7c72749ba7f482'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:34.546770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d6408e91-41e1-415e-9fc1-338372579d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3ca63f8f8484ddf9d7c72749ba7f482'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:34.564948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1247/2479] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:26:34 2022] DELETE /identity/v3/users/b3ca63f8f8484ddf9d7c72749ba7f482 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:34.812054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a755ba-0813-4786-ad72-95e35400387f service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:34.835685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a755ba-0813-4786-ad72-95e35400387f service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:34.954682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be943afb-12f5-4c84-9ef9-b765bc843668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:34.955686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be943afb-12f5-4c84-9ef9-b765bc843668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:34.956128 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be943afb-12f5-4c84-9ef9-b765bc843668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:34.958552 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be943afb-12f5-4c84-9ef9-b765bc843668 admin admin] PATH_INFO: `/v3/projects/ab7deafed6f447faade3e3d8d281617a` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:34.965163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be943afb-12f5-4c84-9ef9-b765bc843668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7deafed6f447faade3e3d8d281617a)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:34.969201 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be943afb-12f5-4c84-9ef9-b765bc843668 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:35.065658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-be943afb-12f5-4c84-9ef9-b765bc843668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7deafed6f447faade3e3d8d281617a', 'request_id': 'req-be943afb-12f5-4c84-9ef9-b765bc843668'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:35.257523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-be943afb-12f5-4c84-9ef9-b765bc843668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7deafed6f447faade3e3d8d281617a', 'request_id': 'req-be943afb-12f5-4c84-9ef9-b765bc843668'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:35.269824 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-be943afb-12f5-4c84-9ef9-b765bc843668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7deafed6f447faade3e3d8d281617a', 'request_id': 'req-be943afb-12f5-4c84-9ef9-b765bc843668'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:35.284731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-be943afb-12f5-4c84-9ef9-b765bc843668 admin admin] The token cache is being invalidate because project ab7deafed6f447faade3e3d8d281617a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:26:35.284981 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-be943afb-12f5-4c84-9ef9-b765bc843668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:35.285634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-be943afb-12f5-4c84-9ef9-b765bc843668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:35.287244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1233/2480] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:26:34 2022] DELETE /identity/v3/projects/ab7deafed6f447faade3e3d8d281617a => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:35.293742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa9c4d-c330-478a-afdf-282122718e27 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:35.294441 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa9c4d-c330-478a-afdf-282122718e27 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:35.294800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa9c4d-c330-478a-afdf-282122718e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:35.309124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:35.323233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-acfa9c4d-c330-478a-afdf-282122718e27 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:35.325857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-acfa9c4d-c330-478a-afdf-282122718e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:35.419729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1248/2481] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:35.429510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1a638-da62-4880-b0a5-ac2d3ad8da3f None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:35.429849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1a638-da62-4880-b0a5-ac2d3ad8da3f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:35.430177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1a638-da62-4880-b0a5-ac2d3ad8da3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:35.451842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-41a1a638-da62-4880-b0a5-ac2d3ad8da3f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:35.453276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-41a1a638-da62-4880-b0a5-ac2d3ad8da3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:35.462510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1234/2482] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:35.466986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa9c4d-c330-478a-afdf-282122718e27 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:35.477429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa9c4d-c330-478a-afdf-282122718e27 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:35.482193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0cd460-4fee-4756-8164-5fd5a389e268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:35.482924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cd460-4fee-4756-8164-5fd5a389e268 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:35.483280 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cd460-4fee-4756-8164-5fd5a389e268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:35.484314 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cd460-4fee-4756-8164-5fd5a389e268 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:35.485121 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0cd460-4fee-4756-8164-5fd5a389e268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:26:35.485497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0cd460-4fee-4756-8164-5fd5a389e268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:35.487381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0cd460-4fee-4756-8164-5fd5a389e268 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:35.500236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1249/2483] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:26:35.505038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1a638-da62-4880-b0a5-ac2d3ad8da3f None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:35.515655 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1a638-da62-4880-b0a5-ac2d3ad8da3f None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:35.522376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668c2f7-9ad5-4729-bd8d-3b6d3ac2dd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:35.522818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668c2f7-9ad5-4729-bd8d-3b6d3ac2dd01 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:35.523069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668c2f7-9ad5-4729-bd8d-3b6d3ac2dd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:35.523400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668c2f7-9ad5-4729-bd8d-3b6d3ac2dd01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:35.524188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668c2f7-9ad5-4729-bd8d-3b6d3ac2dd01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:35.526323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668c2f7-9ad5-4729-bd8d-3b6d3ac2dd01 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:35.577783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1235/2484] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:26:35 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:26:35.582648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0cd460-4fee-4756-8164-5fd5a389e268 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:35.594904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0cd460-4fee-4756-8164-5fd5a389e268 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:35.667808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2b325-c35f-4c87-96f6-b6449328879f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:35.668514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2b325-c35f-4c87-96f6-b6449328879f admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:35.668831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2b325-c35f-4c87-96f6-b6449328879f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:35.669114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2b325-c35f-4c87-96f6-b6449328879f admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:35.669817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2b325-c35f-4c87-96f6-b6449328879f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:35.672656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2b325-c35f-4c87-96f6-b6449328879f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:35.694993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1250/2485] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:26:35 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:26:35.699120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668c2f7-9ad5-4729-bd8d-3b6d3ac2dd01 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:35.710512 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668c2f7-9ad5-4729-bd8d-3b6d3ac2dd01 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:35.717040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9118dbf-35cf-4e63-82b8-ef608e1ea7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:35.718566 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9118dbf-35cf-4e63-82b8-ef608e1ea7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:35.719016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9118dbf-35cf-4e63-82b8-ef608e1ea7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:35.719463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9118dbf-35cf-4e63-82b8-ef608e1ea7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:35.720266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9118dbf-35cf-4e63-82b8-ef608e1ea7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:35.721938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9118dbf-35cf-4e63-82b8-ef608e1ea7a5 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:35.734801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1236/2486] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:26:35.739410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2b325-c35f-4c87-96f6-b6449328879f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:35.754178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2b325-c35f-4c87-96f6-b6449328879f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:35.759916 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ead38-e60c-4bf7-8d82-08fa1a433bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:35.760819 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ead38-e60c-4bf7-8d82-08fa1a433bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:35.761948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ead38-e60c-4bf7-8d82-08fa1a433bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:35.761948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ead38-e60c-4bf7-8d82-08fa1a433bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:35.762745 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ead38-e60c-4bf7-8d82-08fa1a433bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:35.764644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ead38-e60c-4bf7-8d82-08fa1a433bdf admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:35.781702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1251/2487] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:26:35.782991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9118dbf-35cf-4e63-82b8-ef608e1ea7a5 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:35.788631 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:35.795953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9118dbf-35cf-4e63-82b8-ef608e1ea7a5 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:35.805282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaccc2c-0efb-4efc-ab3e-ab71399fba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:35.806672 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaccc2c-0efb-4efc-ab3e-ab71399fba76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:35.811793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaccc2c-0efb-4efc-ab3e-ab71399fba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:35.812639 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaccc2c-0efb-4efc-ab3e-ab71399fba76 admin admin] PATH_INFO: `/v3/projects/5353209c2aa1438ea797961407c2d31b/users/6d980acea00944bd89ad3d727c031ac9/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:35.837057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaccc2c-0efb-4efc-ab3e-ab71399fba76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5353209c2aa1438ea797961407c2d31b, user_id=6d980acea00944bd89ad3d727c031ac9, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:35.842450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaccc2c-0efb-4efc-ab3e-ab71399fba76 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:35.859905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1237/2488] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:26:35 2022] PUT /identity/v3/projects/5353209c2aa1438ea797961407c2d31b/users/6d980acea00944bd89ad3d727c031ac9/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:35.874615 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe939513-32ae-4410-9e82-7c5b9ac96337 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:35.874986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe939513-32ae-4410-9e82-7c5b9ac96337 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:35.875344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe939513-32ae-4410-9e82-7c5b9ac96337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:35.904503 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-fe939513-32ae-4410-9e82-7c5b9ac96337 None None] MFA Rules not processed for user `6d980acea00944bd89ad3d727c031ac9`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:35.906284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-fe939513-32ae-4410-9e82-7c5b9ac96337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:35.995516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:36.003070 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1252/2489] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:36.014269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaccc2c-0efb-4efc-ab3e-ab71399fba76 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:36.017115 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9eaccc2c-0efb-4efc-ab3e-ab71399fba76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:36.112292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaccc2c-0efb-4efc-ab3e-ab71399fba76 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:36.118529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04642c6c-96c3-45ac-9e88-d84a90b397db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:36.119651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04642c6c-96c3-45ac-9e88-d84a90b397db service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:36.119651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04642c6c-96c3-45ac-9e88-d84a90b397db service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:36.120009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04642c6c-96c3-45ac-9e88-d84a90b397db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:36.130090 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04642c6c-96c3-45ac-9e88-d84a90b397db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:36.131894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04642c6c-96c3-45ac-9e88-d84a90b397db service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:36.144306 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1238/2490] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:43.835989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe939513-32ae-4410-9e82-7c5b9ac96337 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:43.896796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe939513-32ae-4410-9e82-7c5b9ac96337 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:43.901557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:43.902432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:43.902717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:43.902997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] PATH_INFO: `/v3/users/6d980acea00944bd89ad3d727c031ac9` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:43.904232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d980acea00944bd89ad3d727c031ac9)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:43.906208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:43.957958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d980acea00944bd89ad3d727c031ac9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:43.965828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d980acea00944bd89ad3d727c031ac9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:43.966610 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d980acea00944bd89ad3d727c031ac9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:43.983604 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d980acea00944bd89ad3d727c031ac9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:43.984583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d980acea00944bd89ad3d727c031ac9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:44.005356 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1253/2491] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:26:43 2022] DELETE /identity/v3/users/6d980acea00944bd89ad3d727c031ac9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:44.009886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a879a6d-a6c5-449d-a8ff-2d3d677604fc None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:44.010556 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a879a6d-a6c5-449d-a8ff-2d3d677604fc None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:44.010755 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a879a6d-a6c5-449d-a8ff-2d3d677604fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:44.032976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-4a879a6d-a6c5-449d-a8ff-2d3d677604fc None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:44.034292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-4a879a6d-a6c5-449d-a8ff-2d3d677604fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:44.103745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1239/2492] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:44.116111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:44.117776 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:44.186834 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcc462-1e32-46e6-ace4-f3a23d772879 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:44.191989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f95ea-95d2-4853-9be7-9cf56d24c4e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:44.192977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f95ea-95d2-4853-9be7-9cf56d24c4e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:44.193319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f95ea-95d2-4853-9be7-9cf56d24c4e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:44.193655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f95ea-95d2-4853-9be7-9cf56d24c4e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:44.205624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f95ea-95d2-4853-9be7-9cf56d24c4e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:44.207582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f95ea-95d2-4853-9be7-9cf56d24c4e6 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:44.219632 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1254/2493] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:44.743649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a879a6d-a6c5-449d-a8ff-2d3d677604fc None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:44.745276 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-4a879a6d-a6c5-449d-a8ff-2d3d677604fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:44.757724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a879a6d-a6c5-449d-a8ff-2d3d677604fc None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:44.762448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687d1c9-7ff8-45ed-9441-399aee61848b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:44.763509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687d1c9-7ff8-45ed-9441-399aee61848b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:44.763871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687d1c9-7ff8-45ed-9441-399aee61848b admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:44.764253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687d1c9-7ff8-45ed-9441-399aee61848b admin admin] PATH_INFO: `/v3/projects/5353209c2aa1438ea797961407c2d31b` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:44.766054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687d1c9-7ff8-45ed-9441-399aee61848b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5353209c2aa1438ea797961407c2d31b)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:44.768058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687d1c9-7ff8-45ed-9441-399aee61848b admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:44.820942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a687d1c9-7ff8-45ed-9441-399aee61848b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5353209c2aa1438ea797961407c2d31b', 'request_id': 'req-a687d1c9-7ff8-45ed-9441-399aee61848b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:44.830675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a687d1c9-7ff8-45ed-9441-399aee61848b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5353209c2aa1438ea797961407c2d31b', 'request_id': 'req-a687d1c9-7ff8-45ed-9441-399aee61848b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:44.833055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:44.841101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a687d1c9-7ff8-45ed-9441-399aee61848b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5353209c2aa1438ea797961407c2d31b', 'request_id': 'req-a687d1c9-7ff8-45ed-9441-399aee61848b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:44.855031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a687d1c9-7ff8-45ed-9441-399aee61848b admin admin] The token cache is being invalidate because project 5353209c2aa1438ea797961407c2d31b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:26:44.855323 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a687d1c9-7ff8-45ed-9441-399aee61848b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:44.856006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a687d1c9-7ff8-45ed-9441-399aee61848b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:44.857643 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1240/2494] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:26:44 2022] DELETE /identity/v3/projects/5353209c2aa1438ea797961407c2d31b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:44.863193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba31de-d33b-4e6a-900a-ad8b922a6a99 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:44.863697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba31de-d33b-4e6a-900a-ad8b922a6a99 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:44.863794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba31de-d33b-4e6a-900a-ad8b922a6a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:44.882909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-0fba31de-d33b-4e6a-900a-ad8b922a6a99 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:44.884321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-0fba31de-d33b-4e6a-900a-ad8b922a6a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:44.905304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:44.960739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1255/2495] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:26:44.969968 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d5294-c10f-4917-9131-84943928b20b None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:44.970365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d5294-c10f-4917-9131-84943928b20b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:44.970788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d5294-c10f-4917-9131-84943928b20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:44.991763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c45d5294-c10f-4917-9131-84943928b20b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:44.993060 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c45d5294-c10f-4917-9131-84943928b20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:45.001546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1241/2496] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:45.006132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba31de-d33b-4e6a-900a-ad8b922a6a99 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:45.018049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba31de-d33b-4e6a-900a-ad8b922a6a99 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:45.026147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ca737-0f27-4e64-8e6b-74f39ba0a483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:45.026841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca737-0f27-4e64-8e6b-74f39ba0a483 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:45.027153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca737-0f27-4e64-8e6b-74f39ba0a483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:45.027429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca737-0f27-4e64-8e6b-74f39ba0a483 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:45.028329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ca737-0f27-4e64-8e6b-74f39ba0a483 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:26:45.028536 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ca737-0f27-4e64-8e6b-74f39ba0a483 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:45.030172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ca737-0f27-4e64-8e6b-74f39ba0a483 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:45.049369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1256/2497] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:26:45.051453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d5294-c10f-4917-9131-84943928b20b None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:45.061787 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d5294-c10f-4917-9131-84943928b20b None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:45.066750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4ad4a-344e-4422-a35d-ee3babf35e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:45.067669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4ad4a-344e-4422-a35d-ee3babf35e3a admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:45.067964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4ad4a-344e-4422-a35d-ee3babf35e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:45.068262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4ad4a-344e-4422-a35d-ee3babf35e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:45.069724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4ad4a-344e-4422-a35d-ee3babf35e3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:45.071115 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4ad4a-344e-4422-a35d-ee3babf35e3a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:45.107945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1242/2498] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:26:45 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:26:45.112845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ca737-0f27-4e64-8e6b-74f39ba0a483 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:45.124969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ca737-0f27-4e64-8e6b-74f39ba0a483 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:45.198342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901363b-44b0-421e-8e70-3664a7493137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:45.199036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901363b-44b0-421e-8e70-3664a7493137 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:45.199329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901363b-44b0-421e-8e70-3664a7493137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:45.199614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901363b-44b0-421e-8e70-3664a7493137 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:45.200347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901363b-44b0-421e-8e70-3664a7493137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:45.202370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901363b-44b0-421e-8e70-3664a7493137 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:45.229457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1257/2499] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:26:45 2022] POST /identity/v3/users => generated 282 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:26:45.234541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4ad4a-344e-4422-a35d-ee3babf35e3a admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:45.247259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4ad4a-344e-4422-a35d-ee3babf35e3a admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:45.252745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b151d-64fc-41ab-a751-edd0c350ee9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:45.253559 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b151d-64fc-41ab-a751-edd0c350ee9a admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:45.253887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b151d-64fc-41ab-a751-edd0c350ee9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:45.254197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b151d-64fc-41ab-a751-edd0c350ee9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:45.254988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b151d-64fc-41ab-a751-edd0c350ee9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:45.256805 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b151d-64fc-41ab-a751-edd0c350ee9a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:45.270775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1243/2500] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:26:45.274849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901363b-44b0-421e-8e70-3664a7493137 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:45.286176 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901363b-44b0-421e-8e70-3664a7493137 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:45.291188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096d3b7-1531-484c-9de3-1f655b89b21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:45.291853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096d3b7-1531-484c-9de3-1f655b89b21e admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:45.292191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096d3b7-1531-484c-9de3-1f655b89b21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:45.292499 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096d3b7-1531-484c-9de3-1f655b89b21e admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:45.293296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096d3b7-1531-484c-9de3-1f655b89b21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:45.295111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096d3b7-1531-484c-9de3-1f655b89b21e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:45.308657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1258/2501] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:26:45.312827 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b151d-64fc-41ab-a751-edd0c350ee9a admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:45.324370 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b151d-64fc-41ab-a751-edd0c350ee9a admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:45.329440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052835e-3773-4d56-923e-4ee6063bcd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:45.330392 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052835e-3773-4d56-923e-4ee6063bcd15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:45.330701 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052835e-3773-4d56-923e-4ee6063bcd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:45.331006 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052835e-3773-4d56-923e-4ee6063bcd15 admin admin] PATH_INFO: `/v3/projects/4d76d1c9e6964c38bd93a971c8c34ce5/users/e11c755d8e574a1cac45ce0ef854a162/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:45.347726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052835e-3773-4d56-923e-4ee6063bcd15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d76d1c9e6964c38bd93a971c8c34ce5, user_id=e11c755d8e574a1cac45ce0ef854a162, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:45.350658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052835e-3773-4d56-923e-4ee6063bcd15 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:45.365005 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1244/2502] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:26:45 2022] PUT /identity/v3/projects/4d76d1c9e6964c38bd93a971c8c34ce5/users/e11c755d8e574a1cac45ce0ef854a162/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:45.379103 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbebaf4-33ff-4526-baed-ec87cac6b35b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:45.379396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbebaf4-33ff-4526-baed-ec87cac6b35b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:45.379680 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbebaf4-33ff-4526-baed-ec87cac6b35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:45.416013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-2fbebaf4-33ff-4526-baed-ec87cac6b35b None None] MFA Rules not processed for user `e11c755d8e574a1cac45ce0ef854a162`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:45.417641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-2fbebaf4-33ff-4526-baed-ec87cac6b35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:45.497902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1259/2503] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:45 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:45.503226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052835e-3773-4d56-923e-4ee6063bcd15 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:45.573620 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052835e-3773-4d56-923e-4ee6063bcd15 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:45.580448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012dc31-5555-47c7-b2af-08d27c4ff2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:45.581177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012dc31-5555-47c7-b2af-08d27c4ff2d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:45.581492 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012dc31-5555-47c7-b2af-08d27c4ff2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:45.581862 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012dc31-5555-47c7-b2af-08d27c4ff2d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:45.582792 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012dc31-5555-47c7-b2af-08d27c4ff2d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:45.585905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012dc31-5555-47c7-b2af-08d27c4ff2d0 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:45.621534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1245/2504] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:26:45 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:26:45.625546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbebaf4-33ff-4526-baed-ec87cac6b35b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:45.636840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbebaf4-33ff-4526-baed-ec87cac6b35b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:45.655371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:45.719889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfc757-4319-4db3-a482-8594dc6e0bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:45.720776 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfc757-4319-4db3-a482-8594dc6e0bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:45.721276 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfc757-4319-4db3-a482-8594dc6e0bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:45.721602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfc757-4319-4db3-a482-8594dc6e0bbd admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:45.722524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfc757-4319-4db3-a482-8594dc6e0bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:45.725688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfc757-4319-4db3-a482-8594dc6e0bbd admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:45.752114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1260/2505] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:26:45 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:26:45.754279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012dc31-5555-47c7-b2af-08d27c4ff2d0 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:45.772580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012dc31-5555-47c7-b2af-08d27c4ff2d0 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:45.778307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6623a62-7b31-4fbc-9a96-20a99fcac1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:45.779001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6623a62-7b31-4fbc-9a96-20a99fcac1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:45.779291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6623a62-7b31-4fbc-9a96-20a99fcac1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:45.779572 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6623a62-7b31-4fbc-9a96-20a99fcac1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:45.780395 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6623a62-7b31-4fbc-9a96-20a99fcac1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:45.782712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6623a62-7b31-4fbc-9a96-20a99fcac1f3 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:45.803716 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1246/2506] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:26:45.805126 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfc757-4319-4db3-a482-8594dc6e0bbd admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:45.818771 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfc757-4319-4db3-a482-8594dc6e0bbd admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:45.826736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0453775-5653-4b1b-8d26-b4df2920ed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:45.828546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0453775-5653-4b1b-8d26-b4df2920ed16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:45.828990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0453775-5653-4b1b-8d26-b4df2920ed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:45.829426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0453775-5653-4b1b-8d26-b4df2920ed16 admin admin] PATH_INFO: `/v3/projects/a126e1f2c7254691945a16b86a69c04b/users/bda84ad1fd3e482ea2c49d3f2f6c5170/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:45.854556 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0453775-5653-4b1b-8d26-b4df2920ed16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a126e1f2c7254691945a16b86a69c04b, user_id=bda84ad1fd3e482ea2c49d3f2f6c5170, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:45.857390 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0453775-5653-4b1b-8d26-b4df2920ed16 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:45.873969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1261/2507] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:26:45 2022] PUT /identity/v3/projects/a126e1f2c7254691945a16b86a69c04b/users/bda84ad1fd3e482ea2c49d3f2f6c5170/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:45.878614 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6623a62-7b31-4fbc-9a96-20a99fcac1f3 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:45.889351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6623a62-7b31-4fbc-9a96-20a99fcac1f3 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:45.985454 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:45.997527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abddf0f-ba83-4def-9010-51322c34701c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:45.998476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abddf0f-ba83-4def-9010-51322c34701c admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:45.999844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abddf0f-ba83-4def-9010-51322c34701c admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:46.000348 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abddf0f-ba83-4def-9010-51322c34701c admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:46.001117 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abddf0f-ba83-4def-9010-51322c34701c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:46.002770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abddf0f-ba83-4def-9010-51322c34701c admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:46.018874 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1247/2508] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:26:46.020932 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0453775-5653-4b1b-8d26-b4df2920ed16 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:46.033436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0453775-5653-4b1b-8d26-b4df2920ed16 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:46.039017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbde0e2-4a1b-4dcf-9ee8-43a19d96f574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:46.039701 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbde0e2-4a1b-4dcf-9ee8-43a19d96f574 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:46.040107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbde0e2-4a1b-4dcf-9ee8-43a19d96f574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:46.040490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbde0e2-4a1b-4dcf-9ee8-43a19d96f574 admin admin] PATH_INFO: `/v3/projects/a126e1f2c7254691945a16b86a69c04b/users/bda84ad1fd3e482ea2c49d3f2f6c5170/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:46.043752 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbde0e2-4a1b-4dcf-9ee8-43a19d96f574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a126e1f2c7254691945a16b86a69c04b, user_id=bda84ad1fd3e482ea2c49d3f2f6c5170, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:46.045967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbde0e2-4a1b-4dcf-9ee8-43a19d96f574 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:46.068498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1262/2509] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:26:46 2022] PUT /identity/v3/projects/a126e1f2c7254691945a16b86a69c04b/users/bda84ad1fd3e482ea2c49d3f2f6c5170/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:46.078145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9177d6-4e0f-4774-b7a2-7543b0671674 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:46.078638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9177d6-4e0f-4774-b7a2-7543b0671674 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:46.078739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9177d6-4e0f-4774-b7a2-7543b0671674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:46.108621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-7a9177d6-4e0f-4774-b7a2-7543b0671674 None None] MFA Rules not processed for user `bda84ad1fd3e482ea2c49d3f2f6c5170`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:46.110168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7a9177d6-4e0f-4774-b7a2-7543b0671674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:46.203296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1248/2510] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:46.217081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbde0e2-4a1b-4dcf-9ee8-43a19d96f574 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:46.219158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-dcbde0e2-4a1b-4dcf-9ee8-43a19d96f574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:46.291104 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbde0e2-4a1b-4dcf-9ee8-43a19d96f574 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:46.299037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f5b76-83db-4415-be49-5e1b8601b62c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:46.300008 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f5b76-83db-4415-be49-5e1b8601b62c service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:46.300904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f5b76-83db-4415-be49-5e1b8601b62c service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:46.301330 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f5b76-83db-4415-be49-5e1b8601b62c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:46.314958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f5b76-83db-4415-be49-5e1b8601b62c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:46.316918 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f5b76-83db-4415-be49-5e1b8601b62c service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:46.329905 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1263/2511] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:46.428059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9177d6-4e0f-4774-b7a2-7543b0671674 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:46.439282 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9177d6-4e0f-4774-b7a2-7543b0671674 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:46.446952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d98536-f65a-4a36-8c0c-c10e1948950f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:46.447666 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d98536-f65a-4a36-8c0c-c10e1948950f service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:46.447956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d98536-f65a-4a36-8c0c-c10e1948950f service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:46.448269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d98536-f65a-4a36-8c0c-c10e1948950f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:46.516551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d98536-f65a-4a36-8c0c-c10e1948950f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:46.518066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d98536-f65a-4a36-8c0c-c10e1948950f service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:46.527909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1249/2512] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:46 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:49.319679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f5b76-83db-4415-be49-5e1b8601b62c service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:49.391712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f5b76-83db-4415-be49-5e1b8601b62c service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:49.396915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:49.397808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:49.398093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:49.398372 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] PATH_INFO: `/v3/users/e11c755d8e574a1cac45ce0ef854a162` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:49.399575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e11c755d8e574a1cac45ce0ef854a162)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:49.401589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:49.416997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:49.454883 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e11c755d8e574a1cac45ce0ef854a162'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:49.463486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e11c755d8e574a1cac45ce0ef854a162'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:49.464281 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e11c755d8e574a1cac45ce0ef854a162'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:49.477119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e11c755d8e574a1cac45ce0ef854a162'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:49.477759 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e11c755d8e574a1cac45ce0ef854a162'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:49.490398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1264/2513] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:26:49 2022] DELETE /identity/v3/users/e11c755d8e574a1cac45ce0ef854a162 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:49.495302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babb0e2-90f7-4da7-816c-a348b18499d9 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:49.495794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babb0e2-90f7-4da7-816c-a348b18499d9 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:49.495912 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babb0e2-90f7-4da7-816c-a348b18499d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:49.514743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-0babb0e2-90f7-4da7-816c-a348b18499d9 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:49.515997 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-0babb0e2-90f7-4da7-816c-a348b18499d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:49.578044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1250/2514] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:49.588278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:49.589743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:49.649097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ab0d6-1047-4e64-86c8-cf22187400e4 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:49.652841 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc3250-4096-4e34-955f-4dd2acdb4cca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:49.653484 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc3250-4096-4e34-955f-4dd2acdb4cca service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:49.653772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc3250-4096-4e34-955f-4dd2acdb4cca service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:49.654052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc3250-4096-4e34-955f-4dd2acdb4cca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:49.664190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc3250-4096-4e34-955f-4dd2acdb4cca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:49.665641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc3250-4096-4e34-955f-4dd2acdb4cca service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:49.676182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1265/2515] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:49.965212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babb0e2-90f7-4da7-816c-a348b18499d9 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:49.966655 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-0babb0e2-90f7-4da7-816c-a348b18499d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:49.979162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babb0e2-90f7-4da7-816c-a348b18499d9 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:49.985521 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:49.986400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:49.986695 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:49.986974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] PATH_INFO: `/v3/projects/4d76d1c9e6964c38bd93a971c8c34ce5` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:49.988339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d76d1c9e6964c38bd93a971c8c34ce5)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:49.990285 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:50.003844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:50.047723 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d76d1c9e6964c38bd93a971c8c34ce5', 'request_id': 'req-6e936ab4-d3a9-4789-acdf-63769004eb1c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.057377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d76d1c9e6964c38bd93a971c8c34ce5', 'request_id': 'req-6e936ab4-d3a9-4789-acdf-63769004eb1c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.064352 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d76d1c9e6964c38bd93a971c8c34ce5', 'request_id': 'req-6e936ab4-d3a9-4789-acdf-63769004eb1c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.075424 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] The token cache is being invalidate because project 4d76d1c9e6964c38bd93a971c8c34ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:26:50.075750 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.076382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.078488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1251/2516] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:26:49 2022] DELETE /identity/v3/projects/4d76d1c9e6964c38bd93a971c8c34ce5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:50.082765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc3250-4096-4e34-955f-4dd2acdb4cca service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:50.092673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc3250-4096-4e34-955f-4dd2acdb4cca service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:50.152533 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f33a56-876b-449d-96db-267375cc96cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:50.153417 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f33a56-876b-449d-96db-267375cc96cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:50.153711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f33a56-876b-449d-96db-267375cc96cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:50.154005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f33a56-876b-449d-96db-267375cc96cb admin admin] PATH_INFO: `/v3/users/bda84ad1fd3e482ea2c49d3f2f6c5170` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:50.155221 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f33a56-876b-449d-96db-267375cc96cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda84ad1fd3e482ea2c49d3f2f6c5170)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:50.157786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f33a56-876b-449d-96db-267375cc96cb admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:50.200034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:50.210978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d9f33a56-876b-449d-96db-267375cc96cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda84ad1fd3e482ea2c49d3f2f6c5170'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.218957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d9f33a56-876b-449d-96db-267375cc96cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda84ad1fd3e482ea2c49d3f2f6c5170'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.219586 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d9f33a56-876b-449d-96db-267375cc96cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda84ad1fd3e482ea2c49d3f2f6c5170'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.233424 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d9f33a56-876b-449d-96db-267375cc96cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda84ad1fd3e482ea2c49d3f2f6c5170'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.234122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d9f33a56-876b-449d-96db-267375cc96cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda84ad1fd3e482ea2c49d3f2f6c5170'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.246601 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1266/2517] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:26:50 2022] DELETE /identity/v3/users/bda84ad1fd3e482ea2c49d3f2f6c5170 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:50.519992 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:50.521485 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:50.537753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e936ab4-d3a9-4789-acdf-63769004eb1c admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:50.606082 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fc953-a5d6-4f98-969a-94f6244e7ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:50.606850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fc953-a5d6-4f98-969a-94f6244e7ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:50.607132 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fc953-a5d6-4f98-969a-94f6244e7ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:50.607415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fc953-a5d6-4f98-969a-94f6244e7ccb admin admin] PATH_INFO: `/v3/projects/a126e1f2c7254691945a16b86a69c04b` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:50.608673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fc953-a5d6-4f98-969a-94f6244e7ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a126e1f2c7254691945a16b86a69c04b)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:50.610585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fc953-a5d6-4f98-969a-94f6244e7ccb admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:50.663423 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-058fc953-a5d6-4f98-969a-94f6244e7ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a126e1f2c7254691945a16b86a69c04b', 'request_id': 'req-058fc953-a5d6-4f98-969a-94f6244e7ccb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.672963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-058fc953-a5d6-4f98-969a-94f6244e7ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a126e1f2c7254691945a16b86a69c04b', 'request_id': 'req-058fc953-a5d6-4f98-969a-94f6244e7ccb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.681517 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-058fc953-a5d6-4f98-969a-94f6244e7ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a126e1f2c7254691945a16b86a69c04b', 'request_id': 'req-058fc953-a5d6-4f98-969a-94f6244e7ccb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.693976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-058fc953-a5d6-4f98-969a-94f6244e7ccb admin admin] The token cache is being invalidate because project a126e1f2c7254691945a16b86a69c04b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:26:50.694268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-058fc953-a5d6-4f98-969a-94f6244e7ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.694976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-058fc953-a5d6-4f98-969a-94f6244e7ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:50.696621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1252/2518] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:26:50 2022] DELETE /identity/v3/projects/a126e1f2c7254691945a16b86a69c04b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:50.701668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fe3b2-e0cf-4580-9b1f-9727769ece00 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:50.702162 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fe3b2-e0cf-4580-9b1f-9727769ece00 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:50.702264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fe3b2-e0cf-4580-9b1f-9727769ece00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:50.720044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-257fe3b2-e0cf-4580-9b1f-9727769ece00 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:50.721286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-257fe3b2-e0cf-4580-9b1f-9727769ece00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:50.784975 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1267/2519] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:50.793304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b633157-a33a-4fcb-9edb-38b446531a76 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:50.793639 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b633157-a33a-4fcb-9edb-38b446531a76 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:50.793974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b633157-a33a-4fcb-9edb-38b446531a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:50.817094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-0b633157-a33a-4fcb-9edb-38b446531a76 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:50.818426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-0b633157-a33a-4fcb-9edb-38b446531a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:50.827393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1253/2520] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:50.832119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fe3b2-e0cf-4580-9b1f-9727769ece00 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:50.842606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fe3b2-e0cf-4580-9b1f-9727769ece00 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:50.847593 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30256ecf-0fb0-4403-8f2d-6505da222ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:50.848367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30256ecf-0fb0-4403-8f2d-6505da222ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:50.848675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30256ecf-0fb0-4403-8f2d-6505da222ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:50.848979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30256ecf-0fb0-4403-8f2d-6505da222ca3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:50.849782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30256ecf-0fb0-4403-8f2d-6505da222ca3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:26:50.850144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30256ecf-0fb0-4403-8f2d-6505da222ca3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:50.851851 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30256ecf-0fb0-4403-8f2d-6505da222ca3 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:50.865190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1268/2521] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:26:50.870248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b633157-a33a-4fcb-9edb-38b446531a76 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:50.881814 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b633157-a33a-4fcb-9edb-38b446531a76 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:50.886777 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bb673-ba49-402c-b539-bf53798846ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:50.887476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bb673-ba49-402c-b539-bf53798846ca admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:50.887791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bb673-ba49-402c-b539-bf53798846ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:50.888116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bb673-ba49-402c-b539-bf53798846ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:50.888845 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bb673-ba49-402c-b539-bf53798846ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:50.890943 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bb673-ba49-402c-b539-bf53798846ca admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:50.918223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1254/2522] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:26:50 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:26:50.923693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30256ecf-0fb0-4403-8f2d-6505da222ca3 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:50.934082 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30256ecf-0fb0-4403-8f2d-6505da222ca3 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:50.998224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653a92a-54ea-4ae5-9435-ab4e688e55d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:50.998897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653a92a-54ea-4ae5-9435-ab4e688e55d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:50.999194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653a92a-54ea-4ae5-9435-ab4e688e55d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.000123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653a92a-54ea-4ae5-9435-ab4e688e55d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.000815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653a92a-54ea-4ae5-9435-ab4e688e55d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:51.002755 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653a92a-54ea-4ae5-9435-ab4e688e55d4 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:51.026547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1269/2523] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:26:50 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:26:51.029102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bb673-ba49-402c-b539-bf53798846ca admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:51.034512 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:51.041186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bb673-ba49-402c-b539-bf53798846ca admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:51.056254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a939aa-7cb2-4d63-886e-3a736eef9fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:51.057218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a939aa-7cb2-4d63-886e-3a736eef9fde admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:51.057730 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a939aa-7cb2-4d63-886e-3a736eef9fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.058170 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a939aa-7cb2-4d63-886e-3a736eef9fde admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.059237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a939aa-7cb2-4d63-886e-3a736eef9fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:51.061390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a939aa-7cb2-4d63-886e-3a736eef9fde admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:51.074748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1255/2524] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:26:51.079980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653a92a-54ea-4ae5-9435-ab4e688e55d4 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:51.092705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653a92a-54ea-4ae5-9435-ab4e688e55d4 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:51.099487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b931dc-d1f8-41d1-b22e-456385c58e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:51.100279 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b931dc-d1f8-41d1-b22e-456385c58e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:51.100622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b931dc-d1f8-41d1-b22e-456385c58e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.100997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b931dc-d1f8-41d1-b22e-456385c58e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.101891 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b931dc-d1f8-41d1-b22e-456385c58e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:51.103849 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b931dc-d1f8-41d1-b22e-456385c58e78 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:51.117350 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1270/2525] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:26:51.121737 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a939aa-7cb2-4d63-886e-3a736eef9fde admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:51.134037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a939aa-7cb2-4d63-886e-3a736eef9fde admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:51.139749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1205f6d9-0cb1-4831-acf6-6490a9cc79ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:51.140892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205f6d9-0cb1-4831-acf6-6490a9cc79ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:51.141225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205f6d9-0cb1-4831-acf6-6490a9cc79ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.141507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205f6d9-0cb1-4831-acf6-6490a9cc79ff admin admin] PATH_INFO: `/v3/projects/d4834905cfaa4281b2bf294343b0fa87/users/27915ea441de410c8958feb20cfe4b8c/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.159429 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1205f6d9-0cb1-4831-acf6-6490a9cc79ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4834905cfaa4281b2bf294343b0fa87, user_id=27915ea441de410c8958feb20cfe4b8c, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:51.162295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1205f6d9-0cb1-4831-acf6-6490a9cc79ff admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:51.177359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1256/2526] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:26:51 2022] PUT /identity/v3/projects/d4834905cfaa4281b2bf294343b0fa87/users/27915ea441de410c8958feb20cfe4b8c/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:51.188517 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dfee4-1a83-48b1-a3ea-41af8a7a2fd6 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:51.188840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dfee4-1a83-48b1-a3ea-41af8a7a2fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.189239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dfee4-1a83-48b1-a3ea-41af8a7a2fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.217808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-955dfee4-1a83-48b1-a3ea-41af8a7a2fd6 None None] MFA Rules not processed for user `27915ea441de410c8958feb20cfe4b8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:51.219639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-955dfee4-1a83-48b1-a3ea-41af8a7a2fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:51.228625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:51.288035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1271/2527] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:51.292616 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1205f6d9-0cb1-4831-acf6-6490a9cc79ff admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:51.361009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1205f6d9-0cb1-4831-acf6-6490a9cc79ff admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:51.365722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34383cae-3af9-4210-ba63-fe5147db2bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:51.366379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34383cae-3af9-4210-ba63-fe5147db2bea admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:51.366660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34383cae-3af9-4210-ba63-fe5147db2bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.366938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34383cae-3af9-4210-ba63-fe5147db2bea admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.367682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34383cae-3af9-4210-ba63-fe5147db2bea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:51.369564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34383cae-3af9-4210-ba63-fe5147db2bea admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:51.399463 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1257/2528] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:26:51 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:26:51.403702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955dfee4-1a83-48b1-a3ea-41af8a7a2fd6 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:51.415990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955dfee4-1a83-48b1-a3ea-41af8a7a2fd6 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:51.479466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a469dd-7881-4693-a26a-676de93869ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:51.480196 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a469dd-7881-4693-a26a-676de93869ce admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:51.480481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a469dd-7881-4693-a26a-676de93869ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.480760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a469dd-7881-4693-a26a-676de93869ce admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.481445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a469dd-7881-4693-a26a-676de93869ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:51.483386 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a469dd-7881-4693-a26a-676de93869ce admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:51.505293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1272/2529] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:26:51 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:26:51.510128 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34383cae-3af9-4210-ba63-fe5147db2bea admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:51.521932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34383cae-3af9-4210-ba63-fe5147db2bea admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:51.526999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603e61c-afea-4ffb-bed2-855eae4ace47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:51.527724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603e61c-afea-4ffb-bed2-855eae4ace47 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:51.528037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603e61c-afea-4ffb-bed2-855eae4ace47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.529019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603e61c-afea-4ffb-bed2-855eae4ace47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.529753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603e61c-afea-4ffb-bed2-855eae4ace47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:51.532148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603e61c-afea-4ffb-bed2-855eae4ace47 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:51.544227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1258/2530] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:26:51.550701 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a469dd-7881-4693-a26a-676de93869ce admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:51.561843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a469dd-7881-4693-a26a-676de93869ce admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:51.567374 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72730079-fca7-45ee-b555-b5b1c75be2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:51.568307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72730079-fca7-45ee-b555-b5b1c75be2de admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:51.568592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72730079-fca7-45ee-b555-b5b1c75be2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.569483 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72730079-fca7-45ee-b555-b5b1c75be2de admin admin] PATH_INFO: `/v3/projects/9edb32b60772421a83c8de4f84b33f79/users/acfc6543b62144a78a15187afa01718f/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.585351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72730079-fca7-45ee-b555-b5b1c75be2de admin admin] RBAC: Authorizing `identity:create_grant(project_id=9edb32b60772421a83c8de4f84b33f79, user_id=acfc6543b62144a78a15187afa01718f, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:51.588020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72730079-fca7-45ee-b555-b5b1c75be2de admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:51.604500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1273/2531] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:26:51 2022] PUT /identity/v3/projects/9edb32b60772421a83c8de4f84b33f79/users/acfc6543b62144a78a15187afa01718f/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:51.608735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603e61c-afea-4ffb-bed2-855eae4ace47 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:51.620625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603e61c-afea-4ffb-bed2-855eae4ace47 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:51.689529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b0522-fbe4-4546-a5f6-bb7b2dffb6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:51.690291 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0522-fbe4-4546-a5f6-bb7b2dffb6d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:51.691194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0522-fbe4-4546-a5f6-bb7b2dffb6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.691194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0522-fbe4-4546-a5f6-bb7b2dffb6d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.692147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b0522-fbe4-4546-a5f6-bb7b2dffb6d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:51.694293 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b0522-fbe4-4546-a5f6-bb7b2dffb6d4 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:51.706192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1259/2532] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:26:51.710530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72730079-fca7-45ee-b555-b5b1c75be2de admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:51.726024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72730079-fca7-45ee-b555-b5b1c75be2de admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:51.732481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d7ac5-7af1-4ecb-b4bf-bcfeb003add8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:51.733591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d7ac5-7af1-4ecb-b4bf-bcfeb003add8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:51.734011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d7ac5-7af1-4ecb-b4bf-bcfeb003add8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.734485 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d7ac5-7af1-4ecb-b4bf-bcfeb003add8 admin admin] PATH_INFO: `/v3/projects/9edb32b60772421a83c8de4f84b33f79/users/acfc6543b62144a78a15187afa01718f/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.737204 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d7ac5-7af1-4ecb-b4bf-bcfeb003add8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9edb32b60772421a83c8de4f84b33f79, user_id=acfc6543b62144a78a15187afa01718f, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:51.741129 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d7ac5-7af1-4ecb-b4bf-bcfeb003add8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:51.753197 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1274/2533] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:26:51 2022] PUT /identity/v3/projects/9edb32b60772421a83c8de4f84b33f79/users/acfc6543b62144a78a15187afa01718f/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:51.766916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f830685-1eac-4025-a218-e15390b3c387 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:51.767253 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f830685-1eac-4025-a218-e15390b3c387 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.767528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f830685-1eac-4025-a218-e15390b3c387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.794754 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-2f830685-1eac-4025-a218-e15390b3c387 None None] MFA Rules not processed for user `acfc6543b62144a78a15187afa01718f`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:51.795969 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-2f830685-1eac-4025-a218-e15390b3c387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:51.823240 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:51.868963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1260/2534] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:51 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:26:51.880293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d7ac5-7af1-4ecb-b4bf-bcfeb003add8 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:51.882123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-910d7ac5-7af1-4ecb-b4bf-bcfeb003add8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:51.949409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d7ac5-7af1-4ecb-b4bf-bcfeb003add8 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:51.953832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5268570-b67c-4f93-adc4-8b1120cb1bf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:51.954487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5268570-b67c-4f93-adc4-8b1120cb1bf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:51.954796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5268570-b67c-4f93-adc4-8b1120cb1bf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:51.955080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5268570-b67c-4f93-adc4-8b1120cb1bf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:51.968570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5268570-b67c-4f93-adc4-8b1120cb1bf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:51.970051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5268570-b67c-4f93-adc4-8b1120cb1bf6 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:51.980334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1275/2535] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:51 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:53.380017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f830685-1eac-4025-a218-e15390b3c387 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:53.458235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f830685-1eac-4025-a218-e15390b3c387 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:53.462809 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:53.463678 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:53.463960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:53.464389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] PATH_INFO: `/v3/users/27915ea441de410c8958feb20cfe4b8c` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:53.465660 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] RBAC: Authorizing `identity:delete_user(user_id=27915ea441de410c8958feb20cfe4b8c)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:53.467586 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:53.515744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27915ea441de410c8958feb20cfe4b8c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:53.516439 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27915ea441de410c8958feb20cfe4b8c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:53.529317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27915ea441de410c8958feb20cfe4b8c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:53.548017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27915ea441de410c8958feb20cfe4b8c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:53.548936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27915ea441de410c8958feb20cfe4b8c'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:53.563596 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1261/2536] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:26:53 2022] DELETE /identity/v3/users/27915ea441de410c8958feb20cfe4b8c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:53.568935 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dd529-c735-4f54-92f7-9996c2a519e6 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:53.569445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dd529-c735-4f54-92f7-9996c2a519e6 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:53.569656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dd529-c735-4f54-92f7-9996c2a519e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:53.576603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:53.590757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-9b9dd529-c735-4f54-92f7-9996c2a519e6 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:53.592092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-9b9dd529-c735-4f54-92f7-9996c2a519e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:53.656083 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1276/2537] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:53.666994 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:53.668531 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:53.717166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:53.732590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84ca1e-2697-4cc4-99cc-e464ec67c10a admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:53.736903 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76c970-c7c2-41d3-9764-39ce226610b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:53.737564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c970-c7c2-41d3-9764-39ce226610b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:53.737848 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c970-c7c2-41d3-9764-39ce226610b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:53.738127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c970-c7c2-41d3-9764-39ce226610b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:53.750918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76c970-c7c2-41d3-9764-39ce226610b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:53.752459 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76c970-c7c2-41d3-9764-39ce226610b4 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:53.762909 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1262/2538] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:26:54.049919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dd529-c735-4f54-92f7-9996c2a519e6 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:54.051927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-9b9dd529-c735-4f54-92f7-9996c2a519e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:54.065102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dd529-c735-4f54-92f7-9996c2a519e6 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:54.070319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:54.071204 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:54.071490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:54.071828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] PATH_INFO: `/v3/projects/d4834905cfaa4281b2bf294343b0fa87` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:54.073537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4834905cfaa4281b2bf294343b0fa87)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:54.076591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:54.130633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4834905cfaa4281b2bf294343b0fa87', 'request_id': 'req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.136736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4834905cfaa4281b2bf294343b0fa87', 'request_id': 'req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.143686 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4834905cfaa4281b2bf294343b0fa87', 'request_id': 'req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.157581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] The token cache is being invalidate because project d4834905cfaa4281b2bf294343b0fa87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:26:54.157857 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.158559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.160194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1277/2539] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:26:54 2022] DELETE /identity/v3/projects/d4834905cfaa4281b2bf294343b0fa87 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:54.164455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76c970-c7c2-41d3-9764-39ce226610b4 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:54.174509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76c970-c7c2-41d3-9764-39ce226610b4 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:54.233252 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0807c3-0bf6-41db-b02c-41d3bbc80831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:54.234131 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0807c3-0bf6-41db-b02c-41d3bbc80831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:54.234419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0807c3-0bf6-41db-b02c-41d3bbc80831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:54.234697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0807c3-0bf6-41db-b02c-41d3bbc80831 admin admin] PATH_INFO: `/v3/users/acfc6543b62144a78a15187afa01718f` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:54.235913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0807c3-0bf6-41db-b02c-41d3bbc80831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acfc6543b62144a78a15187afa01718f)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:54.237887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0807c3-0bf6-41db-b02c-41d3bbc80831 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:54.286011 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6a0807c3-0bf6-41db-b02c-41d3bbc80831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acfc6543b62144a78a15187afa01718f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.286582 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6a0807c3-0bf6-41db-b02c-41d3bbc80831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acfc6543b62144a78a15187afa01718f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.293627 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6a0807c3-0bf6-41db-b02c-41d3bbc80831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acfc6543b62144a78a15187afa01718f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.306826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6a0807c3-0bf6-41db-b02c-41d3bbc80831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acfc6543b62144a78a15187afa01718f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.307656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6a0807c3-0bf6-41db-b02c-41d3bbc80831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acfc6543b62144a78a15187afa01718f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.321287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1263/2540] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:26:54 2022] DELETE /identity/v3/users/acfc6543b62144a78a15187afa01718f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:54.613615 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:54.615274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:54.629638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f796f-fd6d-417d-9b57-aa3862b3d9bc admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:54.694885 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a8b89-e781-4527-be00-30d98cde5918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:54.695752 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a8b89-e781-4527-be00-30d98cde5918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:54.696040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a8b89-e781-4527-be00-30d98cde5918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:54.696345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a8b89-e781-4527-be00-30d98cde5918 admin admin] PATH_INFO: `/v3/projects/9edb32b60772421a83c8de4f84b33f79` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:54.697667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a8b89-e781-4527-be00-30d98cde5918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9edb32b60772421a83c8de4f84b33f79)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:54.699570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a8b89-e781-4527-be00-30d98cde5918 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:54.737371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:54.753634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-2a4a8b89-e781-4527-be00-30d98cde5918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9edb32b60772421a83c8de4f84b33f79', 'request_id': 'req-2a4a8b89-e781-4527-be00-30d98cde5918'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.759430 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-2a4a8b89-e781-4527-be00-30d98cde5918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9edb32b60772421a83c8de4f84b33f79', 'request_id': 'req-2a4a8b89-e781-4527-be00-30d98cde5918'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.765799 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-2a4a8b89-e781-4527-be00-30d98cde5918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9edb32b60772421a83c8de4f84b33f79', 'request_id': 'req-2a4a8b89-e781-4527-be00-30d98cde5918'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.779126 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-2a4a8b89-e781-4527-be00-30d98cde5918 admin admin] The token cache is being invalidate because project 9edb32b60772421a83c8de4f84b33f79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:26:54.779534 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-2a4a8b89-e781-4527-be00-30d98cde5918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.780186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-2a4a8b89-e781-4527-be00-30d98cde5918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:26:54.781827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1278/2541] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:26:54 2022] DELETE /identity/v3/projects/9edb32b60772421a83c8de4f84b33f79 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:26:54.788584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e646b-bc2f-40a5-9aa9-26b3ed3161f1 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:54.789101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e646b-bc2f-40a5-9aa9-26b3ed3161f1 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:54.789435 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e646b-bc2f-40a5-9aa9-26b3ed3161f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:54.808914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f52e646b-bc2f-40a5-9aa9-26b3ed3161f1 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:54.810051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f52e646b-bc2f-40a5-9aa9-26b3ed3161f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:54.872388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1264/2542] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:54.881337 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea8ca4-2d59-4015-8f97-42f34c085eef None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:54.881631 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea8ca4-2d59-4015-8f97-42f34c085eef None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:54.881909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea8ca4-2d59-4015-8f97-42f34c085eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:54.903076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-9eea8ca4-2d59-4015-8f97-42f34c085eef None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:54.904338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-9eea8ca4-2d59-4015-8f97-42f34c085eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:54.913017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1279/2543] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:54.917303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e646b-bc2f-40a5-9aa9-26b3ed3161f1 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:54.924419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:26:54.928906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e646b-bc2f-40a5-9aa9-26b3ed3161f1 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:54.934574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5d347-74cb-4746-a202-9444f5c8f2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:54.935258 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5d347-74cb-4746-a202-9444f5c8f2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:54.935781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5d347-74cb-4746-a202-9444f5c8f2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:54.936326 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5d347-74cb-4746-a202-9444f5c8f2a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:54.937277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5d347-74cb-4746-a202-9444f5c8f2a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:26:54.937595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5d347-74cb-4746-a202-9444f5c8f2a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:54.939223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5d347-74cb-4746-a202-9444f5c8f2a8 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:54.951400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1265/2544] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:26:54.955823 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea8ca4-2d59-4015-8f97-42f34c085eef None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:54.965402 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea8ca4-2d59-4015-8f97-42f34c085eef None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:54.970608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851e803-c128-4198-806a-ee991f203d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:54.971260 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851e803-c128-4198-806a-ee991f203d47 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:54.971548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851e803-c128-4198-806a-ee991f203d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:54.971824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851e803-c128-4198-806a-ee991f203d47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:54.972584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851e803-c128-4198-806a-ee991f203d47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:54.974513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851e803-c128-4198-806a-ee991f203d47 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:55.002352 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1280/2545] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:26:54 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:26:55.006515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5d347-74cb-4746-a202-9444f5c8f2a8 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:55.016942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5d347-74cb-4746-a202-9444f5c8f2a8 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:55.081148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff63464-9343-4dee-a7dd-c3cc5932cf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:55.081742 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63464-9343-4dee-a7dd-c3cc5932cf11 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:55.082027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63464-9343-4dee-a7dd-c3cc5932cf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:55.082307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63464-9343-4dee-a7dd-c3cc5932cf11 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:55.082999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff63464-9343-4dee-a7dd-c3cc5932cf11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:55.085064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff63464-9343-4dee-a7dd-c3cc5932cf11 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:55.108270 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1266/2546] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:26:55 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:26:55.113445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851e803-c128-4198-806a-ee991f203d47 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:55.124532 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851e803-c128-4198-806a-ee991f203d47 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:55.129863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ae6a9-8b90-410e-a825-b1fb4464e167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:55.130635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ae6a9-8b90-410e-a825-b1fb4464e167 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:55.130922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ae6a9-8b90-410e-a825-b1fb4464e167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:55.131205 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ae6a9-8b90-410e-a825-b1fb4464e167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:55.131945 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ae6a9-8b90-410e-a825-b1fb4464e167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:55.133596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ae6a9-8b90-410e-a825-b1fb4464e167 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:55.144924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1281/2547] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:26:55.150615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff63464-9343-4dee-a7dd-c3cc5932cf11 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:55.163440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff63464-9343-4dee-a7dd-c3cc5932cf11 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:55.168858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ed568-cc44-4856-9a99-3bef8eba1763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:55.169675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ed568-cc44-4856-9a99-3bef8eba1763 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:55.169964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ed568-cc44-4856-9a99-3bef8eba1763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:55.170243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ed568-cc44-4856-9a99-3bef8eba1763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:55.171022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ed568-cc44-4856-9a99-3bef8eba1763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:55.173788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ed568-cc44-4856-9a99-3bef8eba1763 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:55.184958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1267/2548] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:26:55.189483 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ae6a9-8b90-410e-a825-b1fb4464e167 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:55.200803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ae6a9-8b90-410e-a825-b1fb4464e167 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:55.205684 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a700dc-698e-40d2-bd71-0c22ffcb3dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:55.206839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a700dc-698e-40d2-bd71-0c22ffcb3dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:55.207135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a700dc-698e-40d2-bd71-0c22ffcb3dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:55.207406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a700dc-698e-40d2-bd71-0c22ffcb3dd1 admin admin] PATH_INFO: `/v3/projects/de779a4b49df41799b6ebdc982179d48/users/40f7bb7bb1ff4937ae2de21aae6b9795/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:55.223709 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a700dc-698e-40d2-bd71-0c22ffcb3dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de779a4b49df41799b6ebdc982179d48, user_id=40f7bb7bb1ff4937ae2de21aae6b9795, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:55.226598 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a700dc-698e-40d2-bd71-0c22ffcb3dd1 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:55.241781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1282/2549] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:26:55 2022] PUT /identity/v3/projects/de779a4b49df41799b6ebdc982179d48/users/40f7bb7bb1ff4937ae2de21aae6b9795/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:26:55.254271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefdc53-b7d8-427f-a4e8-aaa6407e4d89 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:55.254560 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefdc53-b7d8-427f-a4e8-aaa6407e4d89 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:55.254860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefdc53-b7d8-427f-a4e8-aaa6407e4d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:55.283480 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ccefdc53-b7d8-427f-a4e8-aaa6407e4d89 None None] MFA Rules not processed for user `40f7bb7bb1ff4937ae2de21aae6b9795`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:26:55.284733 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ccefdc53-b7d8-427f-a4e8-aaa6407e4d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:55.351382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1268/2550] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:26:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:26:55.362616 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a700dc-698e-40d2-bd71-0c22ffcb3dd1 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:26:55.365066 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-73a700dc-698e-40d2-bd71-0c22ffcb3dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:26:55.425397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a700dc-698e-40d2-bd71-0c22ffcb3dd1 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:26:55.429182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57754753-894c-4f8e-8a4b-177b853487e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:26:55.429840 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57754753-894c-4f8e-8a4b-177b853487e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:26:55.430128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57754753-894c-4f8e-8a4b-177b853487e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:26:55.430408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57754753-894c-4f8e-8a4b-177b853487e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:26:55.440673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57754753-894c-4f8e-8a4b-177b853487e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:26:55.442325 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57754753-894c-4f8e-8a4b-177b853487e8 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:26:55.451949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1283/2551] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:26:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:03.914842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefdc53-b7d8-427f-a4e8-aaa6407e4d89 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:03.976771 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefdc53-b7d8-427f-a4e8-aaa6407e4d89 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:03.981277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:03.982146 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:03.982428 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:03.982734 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] PATH_INFO: `/v3/users/40f7bb7bb1ff4937ae2de21aae6b9795` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:03.983961 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40f7bb7bb1ff4937ae2de21aae6b9795)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:03.985931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:03.987534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:04.039222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40f7bb7bb1ff4937ae2de21aae6b9795'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:04.039874 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40f7bb7bb1ff4937ae2de21aae6b9795'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:04.046677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40f7bb7bb1ff4937ae2de21aae6b9795'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:04.059037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40f7bb7bb1ff4937ae2de21aae6b9795'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:04.059738 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40f7bb7bb1ff4937ae2de21aae6b9795'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:04.072347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1269/2552] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:27:03 2022] DELETE /identity/v3/users/40f7bb7bb1ff4937ae2de21aae6b9795 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:04.076919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97573a15-e3d9-411b-a2ca-cda34257dcd8 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:04.077411 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97573a15-e3d9-411b-a2ca-cda34257dcd8 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:04.077522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97573a15-e3d9-411b-a2ca-cda34257dcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:04.096202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-97573a15-e3d9-411b-a2ca-cda34257dcd8 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:04.097436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-97573a15-e3d9-411b-a2ca-cda34257dcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:04.147824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:04.159648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1284/2553] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:04.169053 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:04.170558 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:04.232585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e01fe-90cc-41c4-81e9-f1a5fa776141 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:04.236250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468274fa-663f-4e76-acd5-3affa77844f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:04.236901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468274fa-663f-4e76-acd5-3affa77844f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:04.237186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468274fa-663f-4e76-acd5-3affa77844f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:04.237464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468274fa-663f-4e76-acd5-3affa77844f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:04.246864 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468274fa-663f-4e76-acd5-3affa77844f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:04.248335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468274fa-663f-4e76-acd5-3affa77844f2 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:04.259233 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1270/2554] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:04.534656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97573a15-e3d9-411b-a2ca-cda34257dcd8 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:04.536321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-97573a15-e3d9-411b-a2ca-cda34257dcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:04.549634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97573a15-e3d9-411b-a2ca-cda34257dcd8 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:04.555047 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80062af-a0a8-40d9-b362-74b7a24ae9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:04.555997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80062af-a0a8-40d9-b362-74b7a24ae9d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:04.556312 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80062af-a0a8-40d9-b362-74b7a24ae9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:04.557132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80062af-a0a8-40d9-b362-74b7a24ae9d8 admin admin] PATH_INFO: `/v3/projects/de779a4b49df41799b6ebdc982179d48` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:04.557821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80062af-a0a8-40d9-b362-74b7a24ae9d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de779a4b49df41799b6ebdc982179d48)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:04.559889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80062af-a0a8-40d9-b362-74b7a24ae9d8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:04.620985 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c80062af-a0a8-40d9-b362-74b7a24ae9d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de779a4b49df41799b6ebdc982179d48', 'request_id': 'req-c80062af-a0a8-40d9-b362-74b7a24ae9d8'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:04.624515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c80062af-a0a8-40d9-b362-74b7a24ae9d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de779a4b49df41799b6ebdc982179d48', 'request_id': 'req-c80062af-a0a8-40d9-b362-74b7a24ae9d8'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:04.632836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c80062af-a0a8-40d9-b362-74b7a24ae9d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de779a4b49df41799b6ebdc982179d48', 'request_id': 'req-c80062af-a0a8-40d9-b362-74b7a24ae9d8'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:04.646474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c80062af-a0a8-40d9-b362-74b7a24ae9d8 admin admin] The token cache is being invalidate because project de779a4b49df41799b6ebdc982179d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:27:04.646770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c80062af-a0a8-40d9-b362-74b7a24ae9d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:04.647415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c80062af-a0a8-40d9-b362-74b7a24ae9d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:04.649040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1285/2555] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:27:04 2022] DELETE /identity/v3/projects/de779a4b49df41799b6ebdc982179d48 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:04.654035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677e896-c28d-4845-8508-c9783c09e8c5 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:04.654269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677e896-c28d-4845-8508-c9783c09e8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:04.654551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677e896-c28d-4845-8508-c9783c09e8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:04.683296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f677e896-c28d-4845-8508-c9783c09e8c5 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:04.684879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f677e896-c28d-4845-8508-c9783c09e8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:04.760104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1271/2556] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:04.768519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a98d8f-a886-4f84-a0f9-92db4113f88f None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:04.768821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a98d8f-a886-4f84-a0f9-92db4113f88f None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:04.769106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a98d8f-a886-4f84-a0f9-92db4113f88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:04.787930 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-55a98d8f-a886-4f84-a0f9-92db4113f88f None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:04.789207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-55a98d8f-a886-4f84-a0f9-92db4113f88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:04.797595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1286/2557] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:04.802214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677e896-c28d-4845-8508-c9783c09e8c5 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:04.812182 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677e896-c28d-4845-8508-c9783c09e8c5 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:04.816697 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d68c76-a527-4336-939c-0a09a297541a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:04.817359 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d68c76-a527-4336-939c-0a09a297541a admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:04.817646 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d68c76-a527-4336-939c-0a09a297541a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:04.817952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d68c76-a527-4336-939c-0a09a297541a admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:04.818686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d68c76-a527-4336-939c-0a09a297541a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:27:04.818994 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d68c76-a527-4336-939c-0a09a297541a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:04.820650 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d68c76-a527-4336-939c-0a09a297541a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:04.834193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1272/2558] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:27:04.838360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a98d8f-a886-4f84-a0f9-92db4113f88f None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:04.849172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a98d8f-a886-4f84-a0f9-92db4113f88f None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:04.854056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57533b43-2c1d-4f0d-af01-98564dcd107f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:04.854850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57533b43-2c1d-4f0d-af01-98564dcd107f admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:04.855228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57533b43-2c1d-4f0d-af01-98564dcd107f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:04.855436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57533b43-2c1d-4f0d-af01-98564dcd107f admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:04.856404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57533b43-2c1d-4f0d-af01-98564dcd107f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:04.858486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57533b43-2c1d-4f0d-af01-98564dcd107f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:04.889143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1287/2559] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:27:04 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:27:04.894472 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d68c76-a527-4336-939c-0a09a297541a admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:04.904976 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d68c76-a527-4336-939c-0a09a297541a admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:04.910826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:04.973316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ad146-7f44-475b-b521-69197c6ecd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:04.974091 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ad146-7f44-475b-b521-69197c6ecd6f admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:04.974344 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ad146-7f44-475b-b521-69197c6ecd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:04.974630 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ad146-7f44-475b-b521-69197c6ecd6f admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:04.975363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ad146-7f44-475b-b521-69197c6ecd6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:04.977386 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ad146-7f44-475b-b521-69197c6ecd6f admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:04.996752 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1273/2560] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:27:04 2022] POST /identity/v3/users => generated 280 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:27:05.002045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57533b43-2c1d-4f0d-af01-98564dcd107f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:05.013384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57533b43-2c1d-4f0d-af01-98564dcd107f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:05.018696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570220c-aeb1-4462-abcd-059ec87b85dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:05.019378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570220c-aeb1-4462-abcd-059ec87b85dc admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:05.019661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570220c-aeb1-4462-abcd-059ec87b85dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:05.019946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570220c-aeb1-4462-abcd-059ec87b85dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:05.020728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570220c-aeb1-4462-abcd-059ec87b85dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:05.022348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570220c-aeb1-4462-abcd-059ec87b85dc admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:05.034085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1288/2561] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:27:05.038266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ad146-7f44-475b-b521-69197c6ecd6f admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:05.048804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ad146-7f44-475b-b521-69197c6ecd6f admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:05.053519 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451aa4bf-9713-4e7a-a0c8-a16b5db3928a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:05.054183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451aa4bf-9713-4e7a-a0c8-a16b5db3928a admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:05.054474 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451aa4bf-9713-4e7a-a0c8-a16b5db3928a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:05.054757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451aa4bf-9713-4e7a-a0c8-a16b5db3928a admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:05.055513 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451aa4bf-9713-4e7a-a0c8-a16b5db3928a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:05.057185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451aa4bf-9713-4e7a-a0c8-a16b5db3928a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:05.067835 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1274/2562] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:27:05.072236 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570220c-aeb1-4462-abcd-059ec87b85dc admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:05.082764 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570220c-aeb1-4462-abcd-059ec87b85dc admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:05.087441 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e252a1-a5ca-44ec-b280-43394e571e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:05.088352 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e252a1-a5ca-44ec-b280-43394e571e75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:05.088635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e252a1-a5ca-44ec-b280-43394e571e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:05.088909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e252a1-a5ca-44ec-b280-43394e571e75 admin admin] PATH_INFO: `/v3/projects/eb91cf222e574fa5b9df46f50f8ff4f5/users/b79069b9051e47e68daa494fb86f5f3f/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:05.103338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e252a1-a5ca-44ec-b280-43394e571e75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb91cf222e574fa5b9df46f50f8ff4f5, user_id=b79069b9051e47e68daa494fb86f5f3f, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:05.106026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e252a1-a5ca-44ec-b280-43394e571e75 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:05.118754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1289/2563] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:27:05 2022] PUT /identity/v3/projects/eb91cf222e574fa5b9df46f50f8ff4f5/users/b79069b9051e47e68daa494fb86f5f3f/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:05.129367 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156387f3-1a98-412f-998b-cc9a9570fd1d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:05.129659 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156387f3-1a98-412f-998b-cc9a9570fd1d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:05.129942 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156387f3-1a98-412f-998b-cc9a9570fd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:05.156565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-156387f3-1a98-412f-998b-cc9a9570fd1d None None] MFA Rules not processed for user `b79069b9051e47e68daa494fb86f5f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:05.157707 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-156387f3-1a98-412f-998b-cc9a9570fd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:05.223193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1275/2564] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:05 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:05.254301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e252a1-a5ca-44ec-b280-43394e571e75 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:05.256033 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b0e252a1-a5ca-44ec-b280-43394e571e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:05.313878 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e252a1-a5ca-44ec-b280-43394e571e75 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:05.317581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67346dac-a01d-41bb-95c1-99da104c3700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:05.318232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67346dac-a01d-41bb-95c1-99da104c3700 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:05.318514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67346dac-a01d-41bb-95c1-99da104c3700 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:05.318791 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67346dac-a01d-41bb-95c1-99da104c3700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:05.327064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67346dac-a01d-41bb-95c1-99da104c3700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:05.329070 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67346dac-a01d-41bb-95c1-99da104c3700 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:05.337675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1290/2565] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:05 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:05.351794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156387f3-1a98-412f-998b-cc9a9570fd1d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:05.414704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156387f3-1a98-412f-998b-cc9a9570fd1d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:05.419769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:05.420682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:05.420969 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:05.421244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] PATH_INFO: `/v3/users/b79069b9051e47e68daa494fb86f5f3f` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:05.423271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b79069b9051e47e68daa494fb86f5f3f)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:05.425336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:05.447119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:05.480383 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b79069b9051e47e68daa494fb86f5f3f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:05.481354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b79069b9051e47e68daa494fb86f5f3f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:05.490979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b79069b9051e47e68daa494fb86f5f3f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:05.503696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b79069b9051e47e68daa494fb86f5f3f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:05.504465 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b79069b9051e47e68daa494fb86f5f3f'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:05.518524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1276/2566] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:27:05 2022] DELETE /identity/v3/users/b79069b9051e47e68daa494fb86f5f3f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:05.523144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af2a1c-a9e8-459b-a309-a69adcf20c3e None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:05.523473 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af2a1c-a9e8-459b-a309-a69adcf20c3e None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:05.523758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af2a1c-a9e8-459b-a309-a69adcf20c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:05.540854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-79af2a1c-a9e8-459b-a309-a69adcf20c3e None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:05.542066 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-79af2a1c-a9e8-459b-a309-a69adcf20c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:05.555159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:05.603454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1291/2567] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:05.613573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:05.614996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:05.674322 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd215a46-b510-45b7-87d2-e85751d8e2c7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:05.678078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af339a71-57e9-46a0-aa30-ea37529d00fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:05.678737 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af339a71-57e9-46a0-aa30-ea37529d00fb service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:05.679025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af339a71-57e9-46a0-aa30-ea37529d00fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:05.679315 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af339a71-57e9-46a0-aa30-ea37529d00fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:05.688843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af339a71-57e9-46a0-aa30-ea37529d00fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:05.690281 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af339a71-57e9-46a0-aa30-ea37529d00fb service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:05.699959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1277/2568] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:05.975575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af2a1c-a9e8-459b-a309-a69adcf20c3e None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:05.977132 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-79af2a1c-a9e8-459b-a309-a69adcf20c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:05.989449 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af2a1c-a9e8-459b-a309-a69adcf20c3e None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:05.994370 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1948516-db7a-46b8-a815-625b407c2fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:05.995443 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1948516-db7a-46b8-a815-625b407c2fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:05.995725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1948516-db7a-46b8-a815-625b407c2fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:05.996000 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1948516-db7a-46b8-a815-625b407c2fdc admin admin] PATH_INFO: `/v3/projects/eb91cf222e574fa5b9df46f50f8ff4f5` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:05.997355 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1948516-db7a-46b8-a815-625b407c2fdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb91cf222e574fa5b9df46f50f8ff4f5)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:05.999328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1948516-db7a-46b8-a815-625b407c2fdc admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:06.047977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c1948516-db7a-46b8-a815-625b407c2fdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb91cf222e574fa5b9df46f50f8ff4f5', 'request_id': 'req-c1948516-db7a-46b8-a815-625b407c2fdc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:06.053719 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c1948516-db7a-46b8-a815-625b407c2fdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb91cf222e574fa5b9df46f50f8ff4f5', 'request_id': 'req-c1948516-db7a-46b8-a815-625b407c2fdc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:06.060279 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c1948516-db7a-46b8-a815-625b407c2fdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb91cf222e574fa5b9df46f50f8ff4f5', 'request_id': 'req-c1948516-db7a-46b8-a815-625b407c2fdc'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:06.073810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c1948516-db7a-46b8-a815-625b407c2fdc admin admin] The token cache is being invalidate because project eb91cf222e574fa5b9df46f50f8ff4f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:27:06.074100 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c1948516-db7a-46b8-a815-625b407c2fdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:06.074743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c1948516-db7a-46b8-a815-625b407c2fdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:06.076373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1292/2569] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:27:05 2022] DELETE /identity/v3/projects/eb91cf222e574fa5b9df46f50f8ff4f5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:06.081924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa170ed-663a-4678-a290-8168acea6679 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:06.082441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa170ed-663a-4678-a290-8168acea6679 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:06.082543 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa170ed-663a-4678-a290-8168acea6679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:06.108585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-9fa170ed-663a-4678-a290-8168acea6679 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:06.110075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9fa170ed-663a-4678-a290-8168acea6679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:06.175281 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1278/2570] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:06.183987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b101d3b-4b68-4311-99ac-5df4472c8c1d None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:06.184555 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b101d3b-4b68-4311-99ac-5df4472c8c1d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:06.184681 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b101d3b-4b68-4311-99ac-5df4472c8c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:06.203578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6b101d3b-4b68-4311-99ac-5df4472c8c1d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:06.204818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6b101d3b-4b68-4311-99ac-5df4472c8c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:06.213083 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1293/2571] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:06.217273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa170ed-663a-4678-a290-8168acea6679 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:06.227955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa170ed-663a-4678-a290-8168acea6679 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:06.232767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d4f5c-6edd-4abf-9100-b46b066b5295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:06.233450 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d4f5c-6edd-4abf-9100-b46b066b5295 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:06.233736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d4f5c-6edd-4abf-9100-b46b066b5295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:06.234550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d4f5c-6edd-4abf-9100-b46b066b5295 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:06.235286 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d4f5c-6edd-4abf-9100-b46b066b5295 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:27:06.235599 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d4f5c-6edd-4abf-9100-b46b066b5295 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:06.237244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d4f5c-6edd-4abf-9100-b46b066b5295 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:06.250686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1279/2572] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:27:06.255452 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b101d3b-4b68-4311-99ac-5df4472c8c1d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:06.264859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b101d3b-4b68-4311-99ac-5df4472c8c1d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:06.269633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1dbcc-62c8-4112-b985-133acd590e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:06.270307 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1dbcc-62c8-4112-b985-133acd590e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:06.270588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1dbcc-62c8-4112-b985-133acd590e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:06.270863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1dbcc-62c8-4112-b985-133acd590e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:06.271543 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1dbcc-62c8-4112-b985-133acd590e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:06.273450 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1dbcc-62c8-4112-b985-133acd590e0e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:06.300621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1294/2573] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:27:06 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:27:06.307549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d4f5c-6edd-4abf-9100-b46b066b5295 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:06.318592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d4f5c-6edd-4abf-9100-b46b066b5295 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:06.342835 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:06.377269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6195d79-170a-445d-a73e-d6866e89315b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:06.377943 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6195d79-170a-445d-a73e-d6866e89315b admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:06.378230 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6195d79-170a-445d-a73e-d6866e89315b admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:06.378509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6195d79-170a-445d-a73e-d6866e89315b admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:06.379201 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6195d79-170a-445d-a73e-d6866e89315b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:06.381195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6195d79-170a-445d-a73e-d6866e89315b admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:06.412016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1280/2574] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:27:06 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:27:06.415821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1dbcc-62c8-4112-b985-133acd590e0e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:06.426962 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1dbcc-62c8-4112-b985-133acd590e0e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:06.431550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e706aa5-50b9-4af3-9871-2ac8a03fccc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:06.432244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e706aa5-50b9-4af3-9871-2ac8a03fccc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:06.432528 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e706aa5-50b9-4af3-9871-2ac8a03fccc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:06.432805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e706aa5-50b9-4af3-9871-2ac8a03fccc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:06.433525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e706aa5-50b9-4af3-9871-2ac8a03fccc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:06.435111 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e706aa5-50b9-4af3-9871-2ac8a03fccc3 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:06.446186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1295/2575] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:27:06.450145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6195d79-170a-445d-a73e-d6866e89315b admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:06.460355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6195d79-170a-445d-a73e-d6866e89315b admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:06.464991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013d1fe-d159-4664-b846-0062c2cf9ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:06.465652 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013d1fe-d159-4664-b846-0062c2cf9ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:06.465941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013d1fe-d159-4664-b846-0062c2cf9ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:06.466224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013d1fe-d159-4664-b846-0062c2cf9ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:06.466951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013d1fe-d159-4664-b846-0062c2cf9ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:06.468590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013d1fe-d159-4664-b846-0062c2cf9ce7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:06.478805 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1281/2576] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:27:06.482836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e706aa5-50b9-4af3-9871-2ac8a03fccc3 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:06.493609 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e706aa5-50b9-4af3-9871-2ac8a03fccc3 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:06.498324 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324eea26-97c0-4579-9d50-e2985b077468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:06.499190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324eea26-97c0-4579-9d50-e2985b077468 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:06.499470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324eea26-97c0-4579-9d50-e2985b077468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:06.500315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324eea26-97c0-4579-9d50-e2985b077468 admin admin] PATH_INFO: `/v3/projects/c0ede829ec1840f7a3f28083ae10140d/users/353cb9c4508c437dbacc27b551b4cc69/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:06.514672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324eea26-97c0-4579-9d50-e2985b077468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ede829ec1840f7a3f28083ae10140d, user_id=353cb9c4508c437dbacc27b551b4cc69, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:06.517380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324eea26-97c0-4579-9d50-e2985b077468 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:06.530668 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1296/2577] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:27:06 2022] PUT /identity/v3/projects/c0ede829ec1840f7a3f28083ae10140d/users/353cb9c4508c437dbacc27b551b4cc69/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:06.541726 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d16f0-2261-408f-9454-1fb510c00bb5 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:06.542031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d16f0-2261-408f-9454-1fb510c00bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:06.542302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d16f0-2261-408f-9454-1fb510c00bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:06.569792 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-827d16f0-2261-408f-9454-1fb510c00bb5 None None] MFA Rules not processed for user `353cb9c4508c437dbacc27b551b4cc69`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:06.571017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-827d16f0-2261-408f-9454-1fb510c00bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:06.643763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1282/2578] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:27:06.654922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324eea26-97c0-4579-9d50-e2985b077468 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:06.657464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-324eea26-97c0-4579-9d50-e2985b077468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:06.708165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:06.725897 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324eea26-97c0-4579-9d50-e2985b077468 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:06.729926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911b9db-b035-4c8f-8664-250422847dd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:06.730530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911b9db-b035-4c8f-8664-250422847dd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:06.730817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911b9db-b035-4c8f-8664-250422847dd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:06.731098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911b9db-b035-4c8f-8664-250422847dd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:06.741126 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911b9db-b035-4c8f-8664-250422847dd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:06.742341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911b9db-b035-4c8f-8664-250422847dd2 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:06.751867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1297/2579] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:15.655920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d16f0-2261-408f-9454-1fb510c00bb5 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:15.657411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-827d16f0-2261-408f-9454-1fb510c00bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:15.725648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d16f0-2261-408f-9454-1fb510c00bb5 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:15.729024 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd01d77-c0fe-4d70-8e0c-12bc6f197fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:15.729677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd01d77-c0fe-4d70-8e0c-12bc6f197fdf service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:15.729962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd01d77-c0fe-4d70-8e0c-12bc6f197fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:15.730262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd01d77-c0fe-4d70-8e0c-12bc6f197fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:15.732057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-5dd01d77-c0fe-4d70-8e0c-12bc6f197fdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:15.804402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:15.827785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd01d77-c0fe-4d70-8e0c-12bc6f197fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:15.829825 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd01d77-c0fe-4d70-8e0c-12bc6f197fdf service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:15.843945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1283/2580] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:16.747447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=8 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:19.255747 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911b9db-b035-4c8f-8664-250422847dd2 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:19.260992 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-9911b9db-b035-4c8f-8664-250422847dd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:19.279400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911b9db-b035-4c8f-8664-250422847dd2 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:19.284280 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d0fa6-0d77-4724-9e99-0b073a976f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:19.285149 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d0fa6-0d77-4724-9e99-0b073a976f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:19.285433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d0fa6-0d77-4724-9e99-0b073a976f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:19.285706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d0fa6-0d77-4724-9e99-0b073a976f07 admin admin] PATH_INFO: `/v3/users/31e94cc1c6a743ce898424ad7ba26cf1` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:19.286914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d0fa6-0d77-4724-9e99-0b073a976f07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31e94cc1c6a743ce898424ad7ba26cf1)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:19.288848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d0fa6-0d77-4724-9e99-0b073a976f07 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:19.360487 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8e1d0fa6-0d77-4724-9e99-0b073a976f07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31e94cc1c6a743ce898424ad7ba26cf1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:19.368742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8e1d0fa6-0d77-4724-9e99-0b073a976f07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31e94cc1c6a743ce898424ad7ba26cf1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:19.369619 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8e1d0fa6-0d77-4724-9e99-0b073a976f07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31e94cc1c6a743ce898424ad7ba26cf1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:19.389738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8e1d0fa6-0d77-4724-9e99-0b073a976f07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31e94cc1c6a743ce898424ad7ba26cf1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:19.390352 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8e1d0fa6-0d77-4724-9e99-0b073a976f07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31e94cc1c6a743ce898424ad7ba26cf1'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:19.403478 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1298/2581] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:27:19 2022] DELETE /identity/v3/users/31e94cc1c6a743ce898424ad7ba26cf1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:27:19.596589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd01d77-c0fe-4d70-8e0c-12bc6f197fdf service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:19.606995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd01d77-c0fe-4d70-8e0c-12bc6f197fdf service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:19.671212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86259d95-8972-4ce2-a711-1d4e1ec845e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:19.672682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86259d95-8972-4ce2-a711-1d4e1ec845e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:19.673221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86259d95-8972-4ce2-a711-1d4e1ec845e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:19.673498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86259d95-8972-4ce2-a711-1d4e1ec845e6 admin admin] PATH_INFO: `/v3/projects/277f81ef9ed74a6189221440edd53d4c` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:19.674685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86259d95-8972-4ce2-a711-1d4e1ec845e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=277f81ef9ed74a6189221440edd53d4c)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:19.676649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86259d95-8972-4ce2-a711-1d4e1ec845e6 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:19.724299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-86259d95-8972-4ce2-a711-1d4e1ec845e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277f81ef9ed74a6189221440edd53d4c', 'request_id': 'req-86259d95-8972-4ce2-a711-1d4e1ec845e6'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:19.731738 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-86259d95-8972-4ce2-a711-1d4e1ec845e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277f81ef9ed74a6189221440edd53d4c', 'request_id': 'req-86259d95-8972-4ce2-a711-1d4e1ec845e6'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:19.738965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-86259d95-8972-4ce2-a711-1d4e1ec845e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277f81ef9ed74a6189221440edd53d4c', 'request_id': 'req-86259d95-8972-4ce2-a711-1d4e1ec845e6'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:19.750489 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-86259d95-8972-4ce2-a711-1d4e1ec845e6 admin admin] The token cache is being invalidate because project 277f81ef9ed74a6189221440edd53d4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:27:19.750765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-86259d95-8972-4ce2-a711-1d4e1ec845e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:19.751452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-86259d95-8972-4ce2-a711-1d4e1ec845e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:19.753106 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1284/2582] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:27:19 2022] DELETE /identity/v3/projects/277f81ef9ed74a6189221440edd53d4c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:19.758138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa4ac4-226e-4b23-bc07-ab2da5faa865 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:19.758451 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa4ac4-226e-4b23-bc07-ab2da5faa865 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:19.758738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa4ac4-226e-4b23-bc07-ab2da5faa865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:19.778093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6baa4ac4-226e-4b23-bc07-ab2da5faa865 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:19.779339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6baa4ac4-226e-4b23-bc07-ab2da5faa865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:19.840186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1299/2583] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:19.848707 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464eba3-5a12-4408-a46c-06ee03e2b83d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:19.849196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464eba3-5a12-4408-a46c-06ee03e2b83d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:19.849196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464eba3-5a12-4408-a46c-06ee03e2b83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:19.868955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-0464eba3-5a12-4408-a46c-06ee03e2b83d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:19.870167 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-0464eba3-5a12-4408-a46c-06ee03e2b83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:19.878205 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1285/2584] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:19.882605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa4ac4-226e-4b23-bc07-ab2da5faa865 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:19.892586 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa4ac4-226e-4b23-bc07-ab2da5faa865 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:19.898648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f6f9a-29ee-490a-985c-de0ef6c6dd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:19.899625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f6f9a-29ee-490a-985c-de0ef6c6dd6e admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:19.900188 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f6f9a-29ee-490a-985c-de0ef6c6dd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:19.900685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f6f9a-29ee-490a-985c-de0ef6c6dd6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:19.902368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f6f9a-29ee-490a-985c-de0ef6c6dd6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:27:19.902971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f6f9a-29ee-490a-985c-de0ef6c6dd6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:19.904806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f6f9a-29ee-490a-985c-de0ef6c6dd6e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:19.918125 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1300/2585] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:27:19.922457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464eba3-5a12-4408-a46c-06ee03e2b83d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:19.932827 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464eba3-5a12-4408-a46c-06ee03e2b83d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:19.937747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1950d02-491e-43a6-b1f1-51177e9083c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:19.938397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1950d02-491e-43a6-b1f1-51177e9083c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:19.938685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1950d02-491e-43a6-b1f1-51177e9083c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:19.938990 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1950d02-491e-43a6-b1f1-51177e9083c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:19.939689 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1950d02-491e-43a6-b1f1-51177e9083c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:19.941625 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1950d02-491e-43a6-b1f1-51177e9083c7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:19.967934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1286/2586] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:27:19 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:27:19.972880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f6f9a-29ee-490a-985c-de0ef6c6dd6e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:19.983171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f6f9a-29ee-490a-985c-de0ef6c6dd6e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:20.038799 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02522d-4aa7-4b67-81b9-796ca409946c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:20.039472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02522d-4aa7-4b67-81b9-796ca409946c admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:20.039758 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02522d-4aa7-4b67-81b9-796ca409946c admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:20.040041 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02522d-4aa7-4b67-81b9-796ca409946c admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:20.040770 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02522d-4aa7-4b67-81b9-796ca409946c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:20.042694 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02522d-4aa7-4b67-81b9-796ca409946c admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:20.050136 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:20.067407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1301/2587] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:27:19 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:27:20.068992 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1950d02-491e-43a6-b1f1-51177e9083c7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:20.080615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1950d02-491e-43a6-b1f1-51177e9083c7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:20.086274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ee781-8bcf-4b90-a846-cc27c0258502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:20.087043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ee781-8bcf-4b90-a846-cc27c0258502 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:20.087337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ee781-8bcf-4b90-a846-cc27c0258502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:20.087617 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ee781-8bcf-4b90-a846-cc27c0258502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:20.088390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ee781-8bcf-4b90-a846-cc27c0258502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:20.090013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ee781-8bcf-4b90-a846-cc27c0258502 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:20.101678 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1287/2588] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:27:20.106554 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02522d-4aa7-4b67-81b9-796ca409946c admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:20.117633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02522d-4aa7-4b67-81b9-796ca409946c admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:20.122312 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bf625-bb14-4b55-b65d-57157828f894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:20.122969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bf625-bb14-4b55-b65d-57157828f894 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:20.123261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bf625-bb14-4b55-b65d-57157828f894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:20.123542 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bf625-bb14-4b55-b65d-57157828f894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:20.124293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bf625-bb14-4b55-b65d-57157828f894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:20.125938 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bf625-bb14-4b55-b65d-57157828f894 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:20.136284 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1302/2589] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:27:20.140841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ee781-8bcf-4b90-a846-cc27c0258502 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:20.150863 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ee781-8bcf-4b90-a846-cc27c0258502 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:20.155573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fd411-f777-480d-9461-ee44602c51f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:20.157031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fd411-f777-480d-9461-ee44602c51f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:20.157317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fd411-f777-480d-9461-ee44602c51f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:20.157593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fd411-f777-480d-9461-ee44602c51f9 admin admin] PATH_INFO: `/v3/projects/683554bc1319497db17e58b1b25a5495/users/227b1920a3264295a4a2406c865d571a/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:20.173528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795fd411-f777-480d-9461-ee44602c51f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=683554bc1319497db17e58b1b25a5495, user_id=227b1920a3264295a4a2406c865d571a, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:20.176289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795fd411-f777-480d-9461-ee44602c51f9 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:20.190511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1288/2590] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:27:20 2022] PUT /identity/v3/projects/683554bc1319497db17e58b1b25a5495/users/227b1920a3264295a4a2406c865d571a/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:20.201074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85243a5-3596-4e8e-b931-ba0f0f3eb510 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:20.201368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85243a5-3596-4e8e-b931-ba0f0f3eb510 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:20.201650 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85243a5-3596-4e8e-b931-ba0f0f3eb510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:20.230122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-a85243a5-3596-4e8e-b931-ba0f0f3eb510 None None] MFA Rules not processed for user `227b1920a3264295a4a2406c865d571a`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:20.231341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-a85243a5-3596-4e8e-b931-ba0f0f3eb510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:20.296442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1303/2591] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:27:20.308739 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fd411-f777-480d-9461-ee44602c51f9 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:20.310720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-795fd411-f777-480d-9461-ee44602c51f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:20.324571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:20.375482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fd411-f777-480d-9461-ee44602c51f9 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:20.380189 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bf0f8-2e57-491c-be28-1dcc616d638b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:20.380879 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bf0f8-2e57-491c-be28-1dcc616d638b service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:20.381163 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bf0f8-2e57-491c-be28-1dcc616d638b service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:20.381438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bf0f8-2e57-491c-be28-1dcc616d638b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:20.393385 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bf0f8-2e57-491c-be28-1dcc616d638b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:20.394854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bf0f8-2e57-491c-be28-1dcc616d638b service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:20.405097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1289/2592] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:20 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:25.579518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85243a5-3596-4e8e-b931-ba0f0f3eb510 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:25.580951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-a85243a5-3596-4e8e-b931-ba0f0f3eb510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:25.647131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85243a5-3596-4e8e-b931-ba0f0f3eb510 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:25.652203 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:25.653079 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:25.653368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:25.653674 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] PATH_INFO: `/v3/users/353cb9c4508c437dbacc27b551b4cc69` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:25.654872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=353cb9c4508c437dbacc27b551b4cc69)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:25.657461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:25.706071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '353cb9c4508c437dbacc27b551b4cc69'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:25.714187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '353cb9c4508c437dbacc27b551b4cc69'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:25.714853 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '353cb9c4508c437dbacc27b551b4cc69'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:25.726901 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:25.728312 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '353cb9c4508c437dbacc27b551b4cc69'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:25.729045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '353cb9c4508c437dbacc27b551b4cc69'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:25.743949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1304/2593] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:27:25 2022] DELETE /identity/v3/users/353cb9c4508c437dbacc27b551b4cc69 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:25.748486 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad04e2d-3d1c-45aa-9892-5bec6e9818f4 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:25.748975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad04e2d-3d1c-45aa-9892-5bec6e9818f4 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:25.749164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad04e2d-3d1c-45aa-9892-5bec6e9818f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:25.769402 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-9ad04e2d-3d1c-45aa-9892-5bec6e9818f4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:25.770612 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9ad04e2d-3d1c-45aa-9892-5bec6e9818f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:25.829523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1290/2594] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:25.839234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:25.840737 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:25.899654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d89530-91c7-443f-96d8-3305ecec5600 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:25.903296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8e730-e0f7-4c8e-bee7-9ba65493ee53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:25.903947 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8e730-e0f7-4c8e-bee7-9ba65493ee53 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:25.904272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8e730-e0f7-4c8e-bee7-9ba65493ee53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:25.904553 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8e730-e0f7-4c8e-bee7-9ba65493ee53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:25.913688 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8e730-e0f7-4c8e-bee7-9ba65493ee53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:25.915127 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8e730-e0f7-4c8e-bee7-9ba65493ee53 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:25.924838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1305/2595] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:26.196455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad04e2d-3d1c-45aa-9892-5bec6e9818f4 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:26.198675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9ad04e2d-3d1c-45aa-9892-5bec6e9818f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:26.215483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad04e2d-3d1c-45aa-9892-5bec6e9818f4 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:26.220145 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:26.221102 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:26.221390 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:26.221671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10 admin admin] PATH_INFO: `/v3/projects/c0ede829ec1840f7a3f28083ae10140d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:26.223066 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ede829ec1840f7a3f28083ae10140d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:26.225013 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:26.273194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ede829ec1840f7a3f28083ae10140d', 'request_id': 'req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:26.280529 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ede829ec1840f7a3f28083ae10140d', 'request_id': 'req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:26.288055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ede829ec1840f7a3f28083ae10140d', 'request_id': 'req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:26.299040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10 admin admin] The token cache is being invalidate because project c0ede829ec1840f7a3f28083ae10140d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:27:26.299332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:26.299991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a24a09db-d9a2-4b6e-a4dc-1352d8de0c10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:26.301580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1291/2596] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:27:26 2022] DELETE /identity/v3/projects/c0ede829ec1840f7a3f28083ae10140d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:26.306754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944138f6-3a3e-44fa-ac06-46a51760ae78 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:26.307049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944138f6-3a3e-44fa-ac06-46a51760ae78 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:26.307335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944138f6-3a3e-44fa-ac06-46a51760ae78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:26.325123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-944138f6-3a3e-44fa-ac06-46a51760ae78 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:26.326357 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-944138f6-3a3e-44fa-ac06-46a51760ae78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:26.393294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1306/2597] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:26.401893 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a67551-476b-4c0b-beab-7701672931e0 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:26.402184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a67551-476b-4c0b-beab-7701672931e0 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:26.402468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a67551-476b-4c0b-beab-7701672931e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:26.421105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-e6a67551-476b-4c0b-beab-7701672931e0 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:26.422349 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e6a67551-476b-4c0b-beab-7701672931e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:26.430498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1292/2598] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:26.434584 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944138f6-3a3e-44fa-ac06-46a51760ae78 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:26.444447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944138f6-3a3e-44fa-ac06-46a51760ae78 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:26.449064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870cb59-9146-4da6-aeb7-a9a8f6f90d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:26.449720 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870cb59-9146-4da6-aeb7-a9a8f6f90d84 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:26.450006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870cb59-9146-4da6-aeb7-a9a8f6f90d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:26.450288 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870cb59-9146-4da6-aeb7-a9a8f6f90d84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:26.451016 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870cb59-9146-4da6-aeb7-a9a8f6f90d84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:27:26.451328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870cb59-9146-4da6-aeb7-a9a8f6f90d84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:26.453515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870cb59-9146-4da6-aeb7-a9a8f6f90d84 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:26.464856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1307/2599] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:27:26.469514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a67551-476b-4c0b-beab-7701672931e0 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:26.478788 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a67551-476b-4c0b-beab-7701672931e0 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:26.483491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a8128-f616-4c97-b18f-eee97929dc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:26.484211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a8128-f616-4c97-b18f-eee97929dc6e admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:26.484502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a8128-f616-4c97-b18f-eee97929dc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:26.484781 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a8128-f616-4c97-b18f-eee97929dc6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:26.485510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a8128-f616-4c97-b18f-eee97929dc6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:26.487422 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a8128-f616-4c97-b18f-eee97929dc6e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:26.512798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1293/2600] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:27:26 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:27:26.517522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870cb59-9146-4da6-aeb7-a9a8f6f90d84 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:26.527948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870cb59-9146-4da6-aeb7-a9a8f6f90d84 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:26.588109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97c84e-d898-438d-91f5-138a81351f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:26.588821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97c84e-d898-438d-91f5-138a81351f6c admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:26.589110 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97c84e-d898-438d-91f5-138a81351f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:26.589393 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97c84e-d898-438d-91f5-138a81351f6c admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:26.590118 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97c84e-d898-438d-91f5-138a81351f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:26.592049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97c84e-d898-438d-91f5-138a81351f6c admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:26.614649 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1308/2601] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:27:26 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:27:26.619377 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a8128-f616-4c97-b18f-eee97929dc6e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:26.631033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a8128-f616-4c97-b18f-eee97929dc6e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:26.636035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c0030-035e-43c8-9031-5680151d3eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:26.636841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c0030-035e-43c8-9031-5680151d3eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:26.637240 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c0030-035e-43c8-9031-5680151d3eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:26.637526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c0030-035e-43c8-9031-5680151d3eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:26.638432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c0030-035e-43c8-9031-5680151d3eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:26.640830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c0030-035e-43c8-9031-5680151d3eb8 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:26.643275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:26.656016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1294/2602] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:27:26.659535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97c84e-d898-438d-91f5-138a81351f6c admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:26.670410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97c84e-d898-438d-91f5-138a81351f6c admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:26.675749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21a577-602f-4f57-9222-b8a2fc4145b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:26.676522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21a577-602f-4f57-9222-b8a2fc4145b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:26.676827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21a577-602f-4f57-9222-b8a2fc4145b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:26.677116 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21a577-602f-4f57-9222-b8a2fc4145b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:26.677848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21a577-602f-4f57-9222-b8a2fc4145b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:26.681081 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21a577-602f-4f57-9222-b8a2fc4145b9 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:26.682624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:26.694591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1309/2603] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:27:26.698329 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c0030-035e-43c8-9031-5680151d3eb8 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:26.708841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c0030-035e-43c8-9031-5680151d3eb8 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:26.713446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe0469-6069-4732-a9bb-ab720aa55f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:26.714325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe0469-6069-4732-a9bb-ab720aa55f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:26.714613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe0469-6069-4732-a9bb-ab720aa55f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:26.714896 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe0469-6069-4732-a9bb-ab720aa55f3e admin admin] PATH_INFO: `/v3/projects/cdf8f41da1154044a1b341f40f98371b/users/13a9b5d88514401685f22445eac1a4b2/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:26.729222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe0469-6069-4732-a9bb-ab720aa55f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf8f41da1154044a1b341f40f98371b, user_id=13a9b5d88514401685f22445eac1a4b2, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:26.731883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe0469-6069-4732-a9bb-ab720aa55f3e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:26.745412 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1295/2604] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:27:26 2022] PUT /identity/v3/projects/cdf8f41da1154044a1b341f40f98371b/users/13a9b5d88514401685f22445eac1a4b2/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:26.756175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34c841-8d28-48e4-a4b5-d00cbe61c7e4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:26.756481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34c841-8d28-48e4-a4b5-d00cbe61c7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:26.756779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34c841-8d28-48e4-a4b5-d00cbe61c7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:26.784920 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-5e34c841-8d28-48e4-a4b5-d00cbe61c7e4 None None] MFA Rules not processed for user `13a9b5d88514401685f22445eac1a4b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:26.786181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5e34c841-8d28-48e4-a4b5-d00cbe61c7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:26.854449 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1310/2605] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:26.865970 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe0469-6069-4732-a9bb-ab720aa55f3e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:26.867696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-0afe0469-6069-4732-a9bb-ab720aa55f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:26.925964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe0469-6069-4732-a9bb-ab720aa55f3e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:26.929833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2b702-6826-4067-8471-dcfc16c6d3fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:26.930488 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2b702-6826-4067-8471-dcfc16c6d3fe service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:26.930776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2b702-6826-4067-8471-dcfc16c6d3fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:26.931056 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2b702-6826-4067-8471-dcfc16c6d3fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:26.940280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2b702-6826-4067-8471-dcfc16c6d3fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:26.942049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2b702-6826-4067-8471-dcfc16c6d3fe service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:26.952388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1296/2606] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:28.846189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34c841-8d28-48e4-a4b5-d00cbe61c7e4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:28.847827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5e34c841-8d28-48e4-a4b5-d00cbe61c7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:28.918525 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34c841-8d28-48e4-a4b5-d00cbe61c7e4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:28.923886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:28.924810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:28.925092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:28.925367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] PATH_INFO: `/v3/users/227b1920a3264295a4a2406c865d571a` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:28.926547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=227b1920a3264295a4a2406c865d571a)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:28.928504 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:28.975335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '227b1920a3264295a4a2406c865d571a'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:28.982348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '227b1920a3264295a4a2406c865d571a'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:28.982994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '227b1920a3264295a4a2406c865d571a'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:28.995172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '227b1920a3264295a4a2406c865d571a'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:28.995873 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '227b1920a3264295a4a2406c865d571a'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:29.008161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1311/2607] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:27:28 2022] DELETE /identity/v3/users/227b1920a3264295a4a2406c865d571a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:29.012765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec233c5-dbae-4e5b-a3f9-05e189dbb01a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:29.013296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec233c5-dbae-4e5b-a3f9-05e189dbb01a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:29.013396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec233c5-dbae-4e5b-a3f9-05e189dbb01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:29.031835 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-9ec233c5-dbae-4e5b-a3f9-05e189dbb01a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:29.033077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9ec233c5-dbae-4e5b-a3f9-05e189dbb01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:29.094667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1297/2608] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:29.105744 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:29.107753 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:29.115828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:29.169058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50574471-2063-4a02-821d-da0fb97ddf7b admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:29.173795 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860a0dd-c966-47e6-a386-2e83cbe057a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:29.174442 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860a0dd-c966-47e6-a386-2e83cbe057a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:29.174726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860a0dd-c966-47e6-a386-2e83cbe057a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:29.175006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860a0dd-c966-47e6-a386-2e83cbe057a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:29.184044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860a0dd-c966-47e6-a386-2e83cbe057a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:29.185507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860a0dd-c966-47e6-a386-2e83cbe057a7 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:29.194882 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1312/2609] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:29.476812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec233c5-dbae-4e5b-a3f9-05e189dbb01a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:29.478341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9ec233c5-dbae-4e5b-a3f9-05e189dbb01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:29.505118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec233c5-dbae-4e5b-a3f9-05e189dbb01a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:29.511317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af274d-c6c2-4cac-90c8-b997a7ea9643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:29.512226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af274d-c6c2-4cac-90c8-b997a7ea9643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:29.512515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af274d-c6c2-4cac-90c8-b997a7ea9643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:29.512796 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af274d-c6c2-4cac-90c8-b997a7ea9643 admin admin] PATH_INFO: `/v3/projects/683554bc1319497db17e58b1b25a5495` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:29.513986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af274d-c6c2-4cac-90c8-b997a7ea9643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=683554bc1319497db17e58b1b25a5495)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:29.516494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af274d-c6c2-4cac-90c8-b997a7ea9643 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:29.561993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-23af274d-c6c2-4cac-90c8-b997a7ea9643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683554bc1319497db17e58b1b25a5495', 'request_id': 'req-23af274d-c6c2-4cac-90c8-b997a7ea9643'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:29.569206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-23af274d-c6c2-4cac-90c8-b997a7ea9643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683554bc1319497db17e58b1b25a5495', 'request_id': 'req-23af274d-c6c2-4cac-90c8-b997a7ea9643'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:29.576227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-23af274d-c6c2-4cac-90c8-b997a7ea9643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683554bc1319497db17e58b1b25a5495', 'request_id': 'req-23af274d-c6c2-4cac-90c8-b997a7ea9643'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:29.589563 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:29.594602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-23af274d-c6c2-4cac-90c8-b997a7ea9643 admin admin] The token cache is being invalidate because project 683554bc1319497db17e58b1b25a5495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:27:29.595063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-23af274d-c6c2-4cac-90c8-b997a7ea9643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:29.595800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-23af274d-c6c2-4cac-90c8-b997a7ea9643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:29.598397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1298/2610] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:27:29 2022] DELETE /identity/v3/projects/683554bc1319497db17e58b1b25a5495 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:29.603332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3da02-07ba-4a20-be95-fb68917be5e4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:29.603626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3da02-07ba-4a20-be95-fb68917be5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:29.603908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3da02-07ba-4a20-be95-fb68917be5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:29.622052 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-5be3da02-07ba-4a20-be95-fb68917be5e4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:29.623304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-5be3da02-07ba-4a20-be95-fb68917be5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:29.681548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1313/2611] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:27:29.689945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe219439-cea4-43c8-86ec-feb3e40703cb None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:29.690240 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe219439-cea4-43c8-86ec-feb3e40703cb None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:29.690523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe219439-cea4-43c8-86ec-feb3e40703cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:29.708351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-fe219439-cea4-43c8-86ec-feb3e40703cb None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:29.709552 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-fe219439-cea4-43c8-86ec-feb3e40703cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:29.717358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1299/2612] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:29.721404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3da02-07ba-4a20-be95-fb68917be5e4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:29.731013 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3da02-07ba-4a20-be95-fb68917be5e4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:29.736141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd9780-bc51-4cfc-be6d-8cf598086018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:29.736798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd9780-bc51-4cfc-be6d-8cf598086018 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:29.737089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd9780-bc51-4cfc-be6d-8cf598086018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:29.737374 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd9780-bc51-4cfc-be6d-8cf598086018 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:29.738123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd9780-bc51-4cfc-be6d-8cf598086018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:27:29.738436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd9780-bc51-4cfc-be6d-8cf598086018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:29.740086 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd9780-bc51-4cfc-be6d-8cf598086018 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:29.752801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1314/2613] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:27:29.756261 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe219439-cea4-43c8-86ec-feb3e40703cb None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:29.765334 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe219439-cea4-43c8-86ec-feb3e40703cb None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:29.769991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210b7df-5692-43f8-baee-4830b367956c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:29.770632 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210b7df-5692-43f8-baee-4830b367956c admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:29.770922 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210b7df-5692-43f8-baee-4830b367956c admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:29.771202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210b7df-5692-43f8-baee-4830b367956c admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:29.771884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210b7df-5692-43f8-baee-4830b367956c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:29.773828 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210b7df-5692-43f8-baee-4830b367956c admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:29.800557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1300/2614] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:27:29 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:27:29.804625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd9780-bc51-4cfc-be6d-8cf598086018 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:29.815166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd9780-bc51-4cfc-be6d-8cf598086018 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:29.871520 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe4613-eac4-4c8c-8663-45508b7a8d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:29.873099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe4613-eac4-4c8c-8663-45508b7a8d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:29.873387 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe4613-eac4-4c8c-8663-45508b7a8d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:29.873661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe4613-eac4-4c8c-8663-45508b7a8d1b admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:29.874345 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe4613-eac4-4c8c-8663-45508b7a8d1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:29.877626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe4613-eac4-4c8c-8663-45508b7a8d1b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:29.897810 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1315/2615] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:27:29 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:27:29.905343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210b7df-5692-43f8-baee-4830b367956c admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:29.915977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210b7df-5692-43f8-baee-4830b367956c admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:29.920991 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ee892-a350-4ad1-b0be-b74fd125438a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:29.922230 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ee892-a350-4ad1-b0be-b74fd125438a admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:29.922515 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ee892-a350-4ad1-b0be-b74fd125438a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:29.922795 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ee892-a350-4ad1-b0be-b74fd125438a admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:29.923779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ee892-a350-4ad1-b0be-b74fd125438a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:29.925413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ee892-a350-4ad1-b0be-b74fd125438a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:29.936717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1301/2616] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:27:29.941386 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe4613-eac4-4c8c-8663-45508b7a8d1b admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:29.941721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ee892-a350-4ad1-b0be-b74fd125438a admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:29.943339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-412ee892-a350-4ad1-b0be-b74fd125438a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:29.951660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe4613-eac4-4c8c-8663-45508b7a8d1b admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:29.956337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ee892-a350-4ad1-b0be-b74fd125438a admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:29.956856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09285941-32d0-4c7b-8eea-64b60d071118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:29.957514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09285941-32d0-4c7b-8eea-64b60d071118 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:29.957801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09285941-32d0-4c7b-8eea-64b60d071118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:29.958080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09285941-32d0-4c7b-8eea-64b60d071118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:29.958796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09285941-32d0-4c7b-8eea-64b60d071118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:29.960416 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09285941-32d0-4c7b-8eea-64b60d071118 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:29.961376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478ee7e-f2ea-4311-a44c-fd2441c919c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:29.962259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478ee7e-f2ea-4311-a44c-fd2441c919c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:29.962552 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478ee7e-f2ea-4311-a44c-fd2441c919c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:29.962831 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478ee7e-f2ea-4311-a44c-fd2441c919c3 admin admin] PATH_INFO: `/v3/users/13a9b5d88514401685f22445eac1a4b2` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:29.964035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478ee7e-f2ea-4311-a44c-fd2441c919c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a9b5d88514401685f22445eac1a4b2)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:29.965972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478ee7e-f2ea-4311-a44c-fd2441c919c3 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:29.970550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1316/2617] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:27:29.974665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09285941-32d0-4c7b-8eea-64b60d071118 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:29.984520 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09285941-32d0-4c7b-8eea-64b60d071118 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:29.989865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a1f82-2930-4a64-8c95-d469e1011385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:29.991032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a1f82-2930-4a64-8c95-d469e1011385 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:29.991392 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a1f82-2930-4a64-8c95-d469e1011385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:29.991682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a1f82-2930-4a64-8c95-d469e1011385 admin admin] PATH_INFO: `/v3/projects/eae4f7e102614248ae60a8cd8b58b226/users/97a06f695760469fad2e627d77854050/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.006531 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:30.010978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a1f82-2930-4a64-8c95-d469e1011385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae4f7e102614248ae60a8cd8b58b226, user_id=97a06f695760469fad2e627d77854050, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:30.013741 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6478ee7e-f2ea-4311-a44c-fd2441c919c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a9b5d88514401685f22445eac1a4b2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:30.014282 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a1f82-2930-4a64-8c95-d469e1011385 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:30.014820 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6478ee7e-f2ea-4311-a44c-fd2441c919c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a9b5d88514401685f22445eac1a4b2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:30.022920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6478ee7e-f2ea-4311-a44c-fd2441c919c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a9b5d88514401685f22445eac1a4b2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:30.035344 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6478ee7e-f2ea-4311-a44c-fd2441c919c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a9b5d88514401685f22445eac1a4b2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:30.036110 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-6478ee7e-f2ea-4311-a44c-fd2441c919c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a9b5d88514401685f22445eac1a4b2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:30.036339 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1317/2618] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:27:29 2022] PUT /identity/v3/projects/eae4f7e102614248ae60a8cd8b58b226/users/97a06f695760469fad2e627d77854050/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:27:30.047821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3b5f3-fa8c-45b2-a53e-01b3727a23b4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.048139 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3b5f3-fa8c-45b2-a53e-01b3727a23b4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.048673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3b5f3-fa8c-45b2-a53e-01b3727a23b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.051938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1302/2619] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:27:29 2022] DELETE /identity/v3/users/13a9b5d88514401685f22445eac1a4b2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:27:30.056274 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2d72d-f69c-4700-9361-e8c486643dff None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.056554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2d72d-f69c-4700-9361-e8c486643dff None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.056837 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2d72d-f69c-4700-9361-e8c486643dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.074899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-6ff2d72d-f69c-4700-9361-e8c486643dff None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:30.075066 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-f4d3b5f3-fa8c-45b2-a53e-01b3727a23b4 None None] MFA Rules not processed for user `97a06f695760469fad2e627d77854050`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:30.076178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f4d3b5f3-fa8c-45b2-a53e-01b3727a23b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:30.076317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6ff2d72d-f69c-4700-9361-e8c486643dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:30.134769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1303/2620] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:27:30.141965 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1318/2621] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:27:30.145255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2d72d-f69c-4700-9361-e8c486643dff None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:30.146242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3b5f3-fa8c-45b2-a53e-01b3727a23b4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:30.146379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6ff2d72d-f69c-4700-9361-e8c486643dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:30.147654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f4d3b5f3-fa8c-45b2-a53e-01b3727a23b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:30.161889 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3b5f3-fa8c-45b2-a53e-01b3727a23b4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:30.166697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20aa77-dbd3-45f1-bdf6-eb0e5ca15fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:30.167362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20aa77-dbd3-45f1-bdf6-eb0e5ca15fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.167646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20aa77-dbd3-45f1-bdf6-eb0e5ca15fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.167927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20aa77-dbd3-45f1-bdf6-eb0e5ca15fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.168807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20aa77-dbd3-45f1-bdf6-eb0e5ca15fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:30.170714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20aa77-dbd3-45f1-bdf6-eb0e5ca15fd9 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:30.184262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:30.197098 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1319/2622] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:27:30 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:27:30.200775 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20aa77-dbd3-45f1-bdf6-eb0e5ca15fd9 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:30.209345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2d72d-f69c-4700-9361-e8c486643dff None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:30.261166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d45dd-2435-4677-a138-5e8d6582f219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:30.261818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d45dd-2435-4677-a138-5e8d6582f219 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.261986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20aa77-dbd3-45f1-bdf6-eb0e5ca15fd9 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:30.262094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d45dd-2435-4677-a138-5e8d6582f219 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.262365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d45dd-2435-4677-a138-5e8d6582f219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.267433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecd724-6d88-4670-9605-1c8edd6049c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:30.268252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecd724-6d88-4670-9605-1c8edd6049c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.268641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecd724-6d88-4670-9605-1c8edd6049c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.269028 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecd724-6d88-4670-9605-1c8edd6049c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.270294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecd724-6d88-4670-9605-1c8edd6049c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:30.271866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecd724-6d88-4670-9605-1c8edd6049c4 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:30.272141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d45dd-2435-4677-a138-5e8d6582f219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:30.273563 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d45dd-2435-4677-a138-5e8d6582f219 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:30.283783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1304/2623] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:30.292710 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1320/2624] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:27:30 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:27:30.296749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d45dd-2435-4677-a138-5e8d6582f219 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:30.307436 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d45dd-2435-4677-a138-5e8d6582f219 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:30.312551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a8b70-e38f-4f8f-a1d2-28898f524963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:30.313300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a8b70-e38f-4f8f-a1d2-28898f524963 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.313592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a8b70-e38f-4f8f-a1d2-28898f524963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.314003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a8b70-e38f-4f8f-a1d2-28898f524963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.314809 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a8b70-e38f-4f8f-a1d2-28898f524963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:30.316554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a8b70-e38f-4f8f-a1d2-28898f524963 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:30.332232 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1305/2625] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:27:30.334254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecd724-6d88-4670-9605-1c8edd6049c4 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:30.347085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecd724-6d88-4670-9605-1c8edd6049c4 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:30.352563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89144d-01f1-48d4-bff4-ec98f4bd6a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:30.353501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89144d-01f1-48d4-bff4-ec98f4bd6a6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.353786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89144d-01f1-48d4-bff4-ec98f4bd6a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.354066 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89144d-01f1-48d4-bff4-ec98f4bd6a6a admin admin] PATH_INFO: `/v3/projects/3c11160fbe9447a9bbfd1a77fd22d607/users/c6270bdb175d49889e565c333d564201/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.372319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89144d-01f1-48d4-bff4-ec98f4bd6a6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c11160fbe9447a9bbfd1a77fd22d607, user_id=c6270bdb175d49889e565c333d564201, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:30.377259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89144d-01f1-48d4-bff4-ec98f4bd6a6a admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:30.403254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1321/2626] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:27:30 2022] PUT /identity/v3/projects/3c11160fbe9447a9bbfd1a77fd22d607/users/c6270bdb175d49889e565c333d564201/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:30.407860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a8b70-e38f-4f8f-a1d2-28898f524963 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:30.418967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a8b70-e38f-4f8f-a1d2-28898f524963 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:30.483025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42750c36-79eb-4716-9539-26165afa57f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:30.483758 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42750c36-79eb-4716-9539-26165afa57f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.484615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42750c36-79eb-4716-9539-26165afa57f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.484892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42750c36-79eb-4716-9539-26165afa57f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.485735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42750c36-79eb-4716-9539-26165afa57f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:30.487419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42750c36-79eb-4716-9539-26165afa57f0 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:30.497540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1306/2627] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:27:30.502027 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89144d-01f1-48d4-bff4-ec98f4bd6a6a admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:30.512418 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89144d-01f1-48d4-bff4-ec98f4bd6a6a admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:30.517638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795577e1-f9fa-4e15-8b88-eb4ab0adbee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:30.518521 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795577e1-f9fa-4e15-8b88-eb4ab0adbee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.518811 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795577e1-f9fa-4e15-8b88-eb4ab0adbee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.519084 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795577e1-f9fa-4e15-8b88-eb4ab0adbee3 admin admin] PATH_INFO: `/v3/projects/3c11160fbe9447a9bbfd1a77fd22d607/users/c6270bdb175d49889e565c333d564201/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.521087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795577e1-f9fa-4e15-8b88-eb4ab0adbee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c11160fbe9447a9bbfd1a77fd22d607, user_id=c6270bdb175d49889e565c333d564201, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:30.523768 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795577e1-f9fa-4e15-8b88-eb4ab0adbee3 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:30.534896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1322/2628] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:27:30 2022] PUT /identity/v3/projects/3c11160fbe9447a9bbfd1a77fd22d607/users/c6270bdb175d49889e565c333d564201/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:27:30.544995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d709a-0d69-4f98-998a-83e1351cc574 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.545287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d709a-0d69-4f98-998a-83e1351cc574 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.545564 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d709a-0d69-4f98-998a-83e1351cc574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.572954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-081d709a-0d69-4f98-998a-83e1351cc574 None None] MFA Rules not processed for user `c6270bdb175d49889e565c333d564201`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:30.574161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-081d709a-0d69-4f98-998a-83e1351cc574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:30.596176 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795577e1-f9fa-4e15-8b88-eb4ab0adbee3 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:30.598287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-795577e1-f9fa-4e15-8b88-eb4ab0adbee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:30.611879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:30.652226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1307/2629] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:30 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:30.663775 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d709a-0d69-4f98-998a-83e1351cc574 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:30.665215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-081d709a-0d69-4f98-998a-83e1351cc574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:30.673811 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:30.687748 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795577e1-f9fa-4e15-8b88-eb4ab0adbee3 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:30.695373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e0b03-f08f-479d-8b2a-35d631abbc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:30.697958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e0b03-f08f-479d-8b2a-35d631abbc9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.707651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e0b03-f08f-479d-8b2a-35d631abbc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.707651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e0b03-f08f-479d-8b2a-35d631abbc9d admin admin] PATH_INFO: `/v3/projects/cdf8f41da1154044a1b341f40f98371b` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.707651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e0b03-f08f-479d-8b2a-35d631abbc9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf8f41da1154044a1b341f40f98371b)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:30.707651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e0b03-f08f-479d-8b2a-35d631abbc9d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:30.740982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d709a-0d69-4f98-998a-83e1351cc574 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:30.746621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61d125-9e8a-46d7-9e38-95a368ce3fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:30.747584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61d125-9e8a-46d7-9e38-95a368ce3fbd service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.748016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61d125-9e8a-46d7-9e38-95a368ce3fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.748479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61d125-9e8a-46d7-9e38-95a368ce3fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.763397 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61d125-9e8a-46d7-9e38-95a368ce3fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:30.765246 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61d125-9e8a-46d7-9e38-95a368ce3fbd service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:30.768426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d35e0b03-f08f-479d-8b2a-35d631abbc9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf8f41da1154044a1b341f40f98371b', 'request_id': 'req-d35e0b03-f08f-479d-8b2a-35d631abbc9d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:30.776585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1308/2630] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:30 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:27:30.776818 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d35e0b03-f08f-479d-8b2a-35d631abbc9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf8f41da1154044a1b341f40f98371b', 'request_id': 'req-d35e0b03-f08f-479d-8b2a-35d631abbc9d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:30.787378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d35e0b03-f08f-479d-8b2a-35d631abbc9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf8f41da1154044a1b341f40f98371b', 'request_id': 'req-d35e0b03-f08f-479d-8b2a-35d631abbc9d'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:30.805726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d35e0b03-f08f-479d-8b2a-35d631abbc9d admin admin] The token cache is being invalidate because project cdf8f41da1154044a1b341f40f98371b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:27:30.806142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d35e0b03-f08f-479d-8b2a-35d631abbc9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:30.807044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d35e0b03-f08f-479d-8b2a-35d631abbc9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:27:30.809612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1323/2631] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:27:30 2022] DELETE /identity/v3/projects/cdf8f41da1154044a1b341f40f98371b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:27:30.875103 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61d125-9e8a-46d7-9e38-95a368ce3fbd service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:30.885240 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61d125-9e8a-46d7-9e38-95a368ce3fbd service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:30.950584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e87e9-d1f0-4bb0-a22c-1878387e9018 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:30.951384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e87e9-d1f0-4bb0-a22c-1878387e9018 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:30.951743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e87e9-d1f0-4bb0-a22c-1878387e9018 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:30.952151 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e87e9-d1f0-4bb0-a22c-1878387e9018 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:30.955215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-055e87e9-d1f0-4bb0-a22c-1878387e9018 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:31.029407 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e87e9-d1f0-4bb0-a22c-1878387e9018 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:31.031921 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e87e9-d1f0-4bb0-a22c-1878387e9018 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:31.048713 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1309/2632] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:40.801037 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:41.040555 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=73 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:46.717957 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed0bef-b8a0-424a-808c-9cca17627def None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:46.718948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed0bef-b8a0-424a-808c-9cca17627def None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:46.719133 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed0bef-b8a0-424a-808c-9cca17627def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:46.738072 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c7ed0bef-b8a0-424a-808c-9cca17627def None None] MFA Rules not processed for user `de859eecbef7498289f132a047e6e5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:27:46.739297 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c7ed0bef-b8a0-424a-808c-9cca17627def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:46.797621 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1324/2633] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:27:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:27:47.867501 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e87e9-d1f0-4bb0-a22c-1878387e9018 service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:27:47.869230 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-055e87e9-d1f0-4bb0-a22c-1878387e9018 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:47.945411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e87e9-d1f0-4bb0-a22c-1878387e9018 service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:27:47.951087 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4673762-96fd-4181-9308-44e3952b1c96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8cf02a52d354517b363c0aaf477550c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:27:47.951747 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4673762-96fd-4181-9308-44e3952b1c96 service placement] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:27:47.952037 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4673762-96fd-4181-9308-44e3952b1c96 service placement] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:27:47.952351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4673762-96fd-4181-9308-44e3952b1c96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:27:47.954203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a4673762-96fd-4181-9308-44e3952b1c96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:27:47.971330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4673762-96fd-4181-9308-44e3952b1c96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:27:47.971330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4673762-96fd-4181-9308-44e3952b1c96 service placement] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:27:47.986894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1310/2634] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:27:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:27:56.790847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=25 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:27:57.981621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=32 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:10.035344 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed0bef-b8a0-424a-808c-9cca17627def None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:10.036925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c7ed0bef-b8a0-424a-808c-9cca17627def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:10.104145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed0bef-b8a0-424a-808c-9cca17627def None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:10.109148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:10.110622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:10.110907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:10.111187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] PATH_INFO: `/v3/users/97a06f695760469fad2e627d77854050` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:10.112468 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a06f695760469fad2e627d77854050)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:10.114407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:10.162656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a06f695760469fad2e627d77854050'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:10.175385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a06f695760469fad2e627d77854050'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:10.176161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a06f695760469fad2e627d77854050'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:10.189913 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a06f695760469fad2e627d77854050'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:10.190625 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a06f695760469fad2e627d77854050'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:10.203364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1325/2635] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:10 2022] DELETE /identity/v3/users/97a06f695760469fad2e627d77854050 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:10.207875 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280a16e-3ccb-41a9-9b26-ab0ccf317072 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:10.208657 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280a16e-3ccb-41a9-9b26-ab0ccf317072 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:10.208657 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280a16e-3ccb-41a9-9b26-ab0ccf317072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:10.231093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-d280a16e-3ccb-41a9-9b26-ab0ccf317072 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:10.232338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d280a16e-3ccb-41a9-9b26-ab0ccf317072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:10.301678 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1311/2636] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:10.311654 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:10.313173 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:10.392848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eb5ba-4cbf-470a-adbb-43dc2d7156ae admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:10.397143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bae3b-35bd-435f-9607-fd92fce64b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:10.397827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bae3b-35bd-435f-9607-fd92fce64b31 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:10.398114 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bae3b-35bd-435f-9607-fd92fce64b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:10.398398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bae3b-35bd-435f-9607-fd92fce64b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:10.409071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bae3b-35bd-435f-9607-fd92fce64b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:10.410565 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bae3b-35bd-435f-9607-fd92fce64b31 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:10.420868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1326/2637] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:28:10.718719 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280a16e-3ccb-41a9-9b26-ab0ccf317072 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:10.722880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-d280a16e-3ccb-41a9-9b26-ab0ccf317072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:10.730415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bae3b-35bd-435f-9607-fd92fce64b31 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:10.731899 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7f0bae3b-35bd-435f-9607-fd92fce64b31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:10.734570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280a16e-3ccb-41a9-9b26-ab0ccf317072 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:10.745635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15aebf-09ca-499b-ba78-c30e447cd150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:10.746433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15aebf-09ca-499b-ba78-c30e447cd150 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:10.746710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15aebf-09ca-499b-ba78-c30e447cd150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:10.746995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15aebf-09ca-499b-ba78-c30e447cd150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:10.748361 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bae3b-35bd-435f-9607-fd92fce64b31 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:10.749111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-cc15aebf-09ca-499b-ba78-c30e447cd150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:10.754384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:10.755614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:10.756181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:10.756470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] PATH_INFO: `/v3/projects/eae4f7e102614248ae60a8cd8b58b226` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:10.757798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae4f7e102614248ae60a8cd8b58b226)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:10.760409 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:10.789210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:10.830493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae4f7e102614248ae60a8cd8b58b226', 'request_id': 'req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:10.835413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15aebf-09ca-499b-ba78-c30e447cd150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:10.836906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15aebf-09ca-499b-ba78-c30e447cd150 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:10.837099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae4f7e102614248ae60a8cd8b58b226', 'request_id': 'req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:10.843929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae4f7e102614248ae60a8cd8b58b226', 'request_id': 'req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:10.847724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1312/2638] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:10.865253 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] The token cache is being invalidate because project eae4f7e102614248ae60a8cd8b58b226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:10.865822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:10.866742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:10.869109 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1327/2639] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:10 2022] DELETE /identity/v3/projects/eae4f7e102614248ae60a8cd8b58b226 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:10.874393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15aebf-09ca-499b-ba78-c30e447cd150 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:10.885797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15aebf-09ca-499b-ba78-c30e447cd150 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:10.965979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5468c3-d346-4663-9e02-09cc1fd39e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:10.966929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5468c3-d346-4663-9e02-09cc1fd39e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:10.967216 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5468c3-d346-4663-9e02-09cc1fd39e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:10.967491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5468c3-d346-4663-9e02-09cc1fd39e49 admin admin] PATH_INFO: `/v3/users/c6270bdb175d49889e565c333d564201` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:10.968858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5468c3-d346-4663-9e02-09cc1fd39e49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6270bdb175d49889e565c333d564201)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:10.971366 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5468c3-d346-4663-9e02-09cc1fd39e49 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:11.020302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:11.027830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f5468c3-d346-4663-9e02-09cc1fd39e49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6270bdb175d49889e565c333d564201'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:11.029069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f5468c3-d346-4663-9e02-09cc1fd39e49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6270bdb175d49889e565c333d564201'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:11.038114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f5468c3-d346-4663-9e02-09cc1fd39e49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6270bdb175d49889e565c333d564201'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:11.052648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f5468c3-d346-4663-9e02-09cc1fd39e49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6270bdb175d49889e565c333d564201'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:11.053454 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f5468c3-d346-4663-9e02-09cc1fd39e49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6270bdb175d49889e565c333d564201'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:11.071242 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1313/2640] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:10 2022] DELETE /identity/v3/users/c6270bdb175d49889e565c333d564201 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:28:11.387087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:11.387087 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:11.408801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8d348-6e65-4aea-af06-4e4ba9829d8e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:11.489989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed97d8-d206-4a8e-849e-321f1071b7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:11.490820 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed97d8-d206-4a8e-849e-321f1071b7fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:11.491108 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed97d8-d206-4a8e-849e-321f1071b7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:11.491389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed97d8-d206-4a8e-849e-321f1071b7fd admin admin] PATH_INFO: `/v3/projects/3c11160fbe9447a9bbfd1a77fd22d607` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:11.492739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed97d8-d206-4a8e-849e-321f1071b7fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c11160fbe9447a9bbfd1a77fd22d607)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:11.494729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed97d8-d206-4a8e-849e-321f1071b7fd admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:11.560702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7bed97d8-d206-4a8e-849e-321f1071b7fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c11160fbe9447a9bbfd1a77fd22d607', 'request_id': 'req-7bed97d8-d206-4a8e-849e-321f1071b7fd'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:11.570022 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7bed97d8-d206-4a8e-849e-321f1071b7fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c11160fbe9447a9bbfd1a77fd22d607', 'request_id': 'req-7bed97d8-d206-4a8e-849e-321f1071b7fd'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:11.578014 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7bed97d8-d206-4a8e-849e-321f1071b7fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c11160fbe9447a9bbfd1a77fd22d607', 'request_id': 'req-7bed97d8-d206-4a8e-849e-321f1071b7fd'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:11.594290 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7bed97d8-d206-4a8e-849e-321f1071b7fd admin admin] The token cache is being invalidate because project 3c11160fbe9447a9bbfd1a77fd22d607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:11.594798 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7bed97d8-d206-4a8e-849e-321f1071b7fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:11.595680 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7bed97d8-d206-4a8e-849e-321f1071b7fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:11.597789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1328/2641] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:11 2022] DELETE /identity/v3/projects/3c11160fbe9447a9bbfd1a77fd22d607 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:11.603080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c6c8e-78a9-421c-ae39-cd65b2c1c24a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:11.603615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c6c8e-78a9-421c-ae39-cd65b2c1c24a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:11.603745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c6c8e-78a9-421c-ae39-cd65b2c1c24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:11.623857 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-994c6c8e-78a9-421c-ae39-cd65b2c1c24a None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:11.625562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-994c6c8e-78a9-421c-ae39-cd65b2c1c24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:11.715894 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1314/2642] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:11.725643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabd8ae-3316-4491-9a40-2553d6fb6d23 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:11.726077 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabd8ae-3316-4491-9a40-2553d6fb6d23 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:11.726476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabd8ae-3316-4491-9a40-2553d6fb6d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:11.747669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-3dabd8ae-3316-4491-9a40-2553d6fb6d23 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:11.748846 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3dabd8ae-3316-4491-9a40-2553d6fb6d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:11.765283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1329/2643] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:11.767767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c6c8e-78a9-421c-ae39-cd65b2c1c24a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:11.785764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c6c8e-78a9-421c-ae39-cd65b2c1c24a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:11.794238 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b02bf0-325b-4f50-b561-f166f5f4b169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:11.795410 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b02bf0-325b-4f50-b561-f166f5f4b169 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:11.795930 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b02bf0-325b-4f50-b561-f166f5f4b169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:11.796479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b02bf0-325b-4f50-b561-f166f5f4b169 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:11.798330 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b02bf0-325b-4f50-b561-f166f5f4b169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:28:11.798844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b02bf0-325b-4f50-b561-f166f5f4b169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:11.802081 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b02bf0-325b-4f50-b561-f166f5f4b169 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:11.829299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1315/2644] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:28:11.833879 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabd8ae-3316-4491-9a40-2553d6fb6d23 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:11.843838 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabd8ae-3316-4491-9a40-2553d6fb6d23 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:11.849282 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d906f34-31dc-40db-b8b9-b24b270585f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:11.850060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d906f34-31dc-40db-b8b9-b24b270585f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:11.850353 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d906f34-31dc-40db-b8b9-b24b270585f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:11.850627 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d906f34-31dc-40db-b8b9-b24b270585f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:11.851326 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d906f34-31dc-40db-b8b9-b24b270585f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:11.853319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d906f34-31dc-40db-b8b9-b24b270585f9 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:11.903175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1330/2645] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:28:11 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:11.907970 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b02bf0-325b-4f50-b561-f166f5f4b169 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:11.919963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b02bf0-325b-4f50-b561-f166f5f4b169 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:11.989760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182efb7-64a3-494b-b93a-fc6e0843ab2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:11.990937 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182efb7-64a3-494b-b93a-fc6e0843ab2d admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:11.991262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182efb7-64a3-494b-b93a-fc6e0843ab2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:11.991542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182efb7-64a3-494b-b93a-fc6e0843ab2d admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:11.992302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182efb7-64a3-494b-b93a-fc6e0843ab2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:11.994235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182efb7-64a3-494b-b93a-fc6e0843ab2d admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:12.021477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1316/2646] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:28:11 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:28:12.022693 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d906f34-31dc-40db-b8b9-b24b270585f9 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:12.034104 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d906f34-31dc-40db-b8b9-b24b270585f9 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:12.039032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27f649-2be7-4dd2-84e5-a8a5b2c3fc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:12.039715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27f649-2be7-4dd2-84e5-a8a5b2c3fc0e admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:12.040047 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27f649-2be7-4dd2-84e5-a8a5b2c3fc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:12.040343 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27f649-2be7-4dd2-84e5-a8a5b2c3fc0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:12.041078 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27f649-2be7-4dd2-84e5-a8a5b2c3fc0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:12.042694 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27f649-2be7-4dd2-84e5-a8a5b2c3fc0e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:12.044419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:12.060366 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1331/2647] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:12.063162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182efb7-64a3-494b-b93a-fc6e0843ab2d admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:12.078087 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182efb7-64a3-494b-b93a-fc6e0843ab2d admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:12.093561 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c47d2-567f-40a8-ac74-db5fff2e5e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:12.094778 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c47d2-567f-40a8-ac74-db5fff2e5e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:12.095083 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c47d2-567f-40a8-ac74-db5fff2e5e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:12.095363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c47d2-567f-40a8-ac74-db5fff2e5e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:12.096141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c47d2-567f-40a8-ac74-db5fff2e5e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:12.097767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c47d2-567f-40a8-ac74-db5fff2e5e50 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:12.109041 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1317/2648] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:12.118707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27f649-2be7-4dd2-84e5-a8a5b2c3fc0e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:12.132466 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27f649-2be7-4dd2-84e5-a8a5b2c3fc0e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:12.140462 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfe86c-aecf-4dc8-8674-c2fec21a3807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:12.142156 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfe86c-aecf-4dc8-8674-c2fec21a3807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:12.142156 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfe86c-aecf-4dc8-8674-c2fec21a3807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:12.142156 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfe86c-aecf-4dc8-8674-c2fec21a3807 admin admin] PATH_INFO: `/v3/projects/754d6f6c545f4e70b08cbc2eaa2a6c13/users/0af111c7b2784dee8b38a706431111d8/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:12.164624 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfe86c-aecf-4dc8-8674-c2fec21a3807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=754d6f6c545f4e70b08cbc2eaa2a6c13, user_id=0af111c7b2784dee8b38a706431111d8, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:12.168172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfe86c-aecf-4dc8-8674-c2fec21a3807 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:12.190749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1332/2649] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:12 2022] PUT /identity/v3/projects/754d6f6c545f4e70b08cbc2eaa2a6c13/users/0af111c7b2784dee8b38a706431111d8/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:12.211469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef3940-1464-4293-8567-e6dff18fd2ad None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:12.211954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef3940-1464-4293-8567-e6dff18fd2ad None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:12.212565 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef3940-1464-4293-8567-e6dff18fd2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:12.269325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f4ef3940-1464-4293-8567-e6dff18fd2ad None None] MFA Rules not processed for user `0af111c7b2784dee8b38a706431111d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:12.271307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f4ef3940-1464-4293-8567-e6dff18fd2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:12.296880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:12.424243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1318/2650] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:12.426696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfe86c-aecf-4dc8-8674-c2fec21a3807 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:12.505872 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfe86c-aecf-4dc8-8674-c2fec21a3807 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:12.510973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795adbc5-5575-4342-a8e4-5d90ac23e194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:12.511672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795adbc5-5575-4342-a8e4-5d90ac23e194 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:12.511952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795adbc5-5575-4342-a8e4-5d90ac23e194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:12.512264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795adbc5-5575-4342-a8e4-5d90ac23e194 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:12.512959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795adbc5-5575-4342-a8e4-5d90ac23e194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:12.514902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795adbc5-5575-4342-a8e4-5d90ac23e194 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:12.550235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1333/2651] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:28:12 2022] POST /identity/v3/projects => generated 363 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:12.555243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef3940-1464-4293-8567-e6dff18fd2ad None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:12.583490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef3940-1464-4293-8567-e6dff18fd2ad None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:12.675588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b6776-d332-4bf4-a9e8-ab4c5c35ee9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:12.676643 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b6776-d332-4bf4-a9e8-ab4c5c35ee9e admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:12.676643 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b6776-d332-4bf4-a9e8-ab4c5c35ee9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:12.677019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b6776-d332-4bf4-a9e8-ab4c5c35ee9e admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:12.677550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b6776-d332-4bf4-a9e8-ab4c5c35ee9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:12.679570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b6776-d332-4bf4-a9e8-ab4c5c35ee9e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:12.706161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1319/2652] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:28:12 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:12.707936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795adbc5-5575-4342-a8e4-5d90ac23e194 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:12.719705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795adbc5-5575-4342-a8e4-5d90ac23e194 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:12.725428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a92aa-49b7-45ef-adb3-ace83e376a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:12.726283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a92aa-49b7-45ef-adb3-ace83e376a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:12.726665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a92aa-49b7-45ef-adb3-ace83e376a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:12.726906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a92aa-49b7-45ef-adb3-ace83e376a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:12.727655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a92aa-49b7-45ef-adb3-ace83e376a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:12.729578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a92aa-49b7-45ef-adb3-ace83e376a38 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:12.745189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1334/2653] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:12.749777 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b6776-d332-4bf4-a9e8-ab4c5c35ee9e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:12.761284 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b6776-d332-4bf4-a9e8-ab4c5c35ee9e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:12.766092 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d81a5-b2aa-4c24-bf06-fe0df2f4641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:12.767645 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d81a5-b2aa-4c24-bf06-fe0df2f4641b admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:12.767933 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d81a5-b2aa-4c24-bf06-fe0df2f4641b admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:12.768250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d81a5-b2aa-4c24-bf06-fe0df2f4641b admin admin] PATH_INFO: `/v3/projects/f42e869a309241e3bf0fb6f348fef0e9/users/1cf6216ce2b446d3a5da725311e507e3/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:12.784907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d81a5-b2aa-4c24-bf06-fe0df2f4641b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42e869a309241e3bf0fb6f348fef0e9, user_id=1cf6216ce2b446d3a5da725311e507e3, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:12.787854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d81a5-b2aa-4c24-bf06-fe0df2f4641b admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:12.802694 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1320/2654] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:12 2022] PUT /identity/v3/projects/f42e869a309241e3bf0fb6f348fef0e9/users/1cf6216ce2b446d3a5da725311e507e3/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:28:12.807640 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a92aa-49b7-45ef-adb3-ace83e376a38 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:12.819699 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a92aa-49b7-45ef-adb3-ace83e376a38 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:12.891492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c24f1c-5c77-422f-9978-a41a7841c1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:12.892240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c24f1c-5c77-422f-9978-a41a7841c1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:12.892530 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c24f1c-5c77-422f-9978-a41a7841c1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:12.892813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c24f1c-5c77-422f-9978-a41a7841c1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:12.893568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c24f1c-5c77-422f-9978-a41a7841c1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:12.895950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c24f1c-5c77-422f-9978-a41a7841c1e0 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:12.909788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1335/2655] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:12.912335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d81a5-b2aa-4c24-bf06-fe0df2f4641b admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:12.930047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d81a5-b2aa-4c24-bf06-fe0df2f4641b admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:12.936357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bdb9e-0b2f-4842-ba7d-ad8dd4bb3427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:12.937287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bdb9e-0b2f-4842-ba7d-ad8dd4bb3427 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:12.937574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bdb9e-0b2f-4842-ba7d-ad8dd4bb3427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:12.937853 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bdb9e-0b2f-4842-ba7d-ad8dd4bb3427 admin admin] PATH_INFO: `/v3/projects/f42e869a309241e3bf0fb6f348fef0e9/users/1cf6216ce2b446d3a5da725311e507e3/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:12.940114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bdb9e-0b2f-4842-ba7d-ad8dd4bb3427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42e869a309241e3bf0fb6f348fef0e9, user_id=1cf6216ce2b446d3a5da725311e507e3, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:12.942840 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bdb9e-0b2f-4842-ba7d-ad8dd4bb3427 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:12.954943 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1321/2656] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:12 2022] PUT /identity/v3/projects/f42e869a309241e3bf0fb6f348fef0e9/users/1cf6216ce2b446d3a5da725311e507e3/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:12.966301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c0ae2-8215-4a3e-93d9-bc0cf520e68a None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:12.966595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c0ae2-8215-4a3e-93d9-bc0cf520e68a None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:12.966877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c0ae2-8215-4a3e-93d9-bc0cf520e68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:12.998294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-d35c0ae2-8215-4a3e-93d9-bc0cf520e68a None None] MFA Rules not processed for user `1cf6216ce2b446d3a5da725311e507e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:12.999594 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-d35c0ae2-8215-4a3e-93d9-bc0cf520e68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:13.035800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:13.092760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1336/2657] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:12 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:13.105038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bdb9e-0b2f-4842-ba7d-ad8dd4bb3427 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:13.106752 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9a6bdb9e-0b2f-4842-ba7d-ad8dd4bb3427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:13.170791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bdb9e-0b2f-4842-ba7d-ad8dd4bb3427 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:13.175957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e259-a9dd-45c7-a5cc-5f07fa86797d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:13.176804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e259-a9dd-45c7-a5cc-5f07fa86797d service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:13.177097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e259-a9dd-45c7-a5cc-5f07fa86797d service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:13.177388 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e259-a9dd-45c7-a5cc-5f07fa86797d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:13.187607 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137e259-a9dd-45c7-a5cc-5f07fa86797d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:13.189196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137e259-a9dd-45c7-a5cc-5f07fa86797d service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:13.200398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1322/2658] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:13 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:13.281435 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c0ae2-8215-4a3e-93d9-bc0cf520e68a None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:13.293848 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c0ae2-8215-4a3e-93d9-bc0cf520e68a None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:13.299035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95196acd-f2ca-4b2c-9940-2748db9323ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:13.299726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95196acd-f2ca-4b2c-9940-2748db9323ed service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:13.300405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95196acd-f2ca-4b2c-9940-2748db9323ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:13.300690 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95196acd-f2ca-4b2c-9940-2748db9323ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:13.372418 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95196acd-f2ca-4b2c-9940-2748db9323ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:13.373906 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95196acd-f2ca-4b2c-9940-2748db9323ed service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:13.384700 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1337/2659] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:16.235820 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e259-a9dd-45c7-a5cc-5f07fa86797d service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:16.237292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a137e259-a9dd-45c7-a5cc-5f07fa86797d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:16.250420 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:16.325581 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e259-a9dd-45c7-a5cc-5f07fa86797d service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:16.331531 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d3769-2652-4dae-aa80-0b74c498ff6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:16.332577 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d3769-2652-4dae-aa80-0b74c498ff6c service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:16.333033 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d3769-2652-4dae-aa80-0b74c498ff6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:16.333479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d3769-2652-4dae-aa80-0b74c498ff6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:16.337328 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a30d3769-2652-4dae-aa80-0b74c498ff6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:16.442974 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d3769-2652-4dae-aa80-0b74c498ff6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:16.445215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d3769-2652-4dae-aa80-0b74c498ff6c service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:16.457001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1323/2660] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:19.978455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95196acd-f2ca-4b2c-9940-2748db9323ed service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:19.979972 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-95196acd-f2ca-4b2c-9940-2748db9323ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:20.077269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95196acd-f2ca-4b2c-9940-2748db9323ed service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:20.083777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1f168-256c-4a04-9216-56b6a864f992 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8cf02a52d354517b363c0aaf477550c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:20.084788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1f168-256c-4a04-9216-56b6a864f992 service placement] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:20.085074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1f168-256c-4a04-9216-56b6a864f992 service placement] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:20.085359 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1f168-256c-4a04-9216-56b6a864f992 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:20.087198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-2cf1f168-256c-4a04-9216-56b6a864f992 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:20.101506 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1f168-256c-4a04-9216-56b6a864f992 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:20.103904 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1f168-256c-4a04-9216-56b6a864f992 service placement] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:20.115612 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1338/2661] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:23.163418 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d3769-2652-4dae-aa80-0b74c498ff6c service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:23.176419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d3769-2652-4dae-aa80-0b74c498ff6c service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:23.182478 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:23.183880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:23.184430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:23.185077 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] PATH_INFO: `/v3/users/0af111c7b2784dee8b38a706431111d8` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:23.187766 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0af111c7b2784dee8b38a706431111d8)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:23.190357 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:23.267812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0af111c7b2784dee8b38a706431111d8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:23.272199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0af111c7b2784dee8b38a706431111d8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:23.293179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0af111c7b2784dee8b38a706431111d8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:23.321457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0af111c7b2784dee8b38a706431111d8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:23.322854 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0af111c7b2784dee8b38a706431111d8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:23.355364 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1324/2662] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:23 2022] DELETE /identity/v3/users/0af111c7b2784dee8b38a706431111d8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:23.359657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6fec5-5851-4d91-9596-9a529f6142bb None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:23.360292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6fec5-5851-4d91-9596-9a529f6142bb None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:23.360292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6fec5-5851-4d91-9596-9a529f6142bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:23.402595 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-8aa6fec5-5851-4d91-9596-9a529f6142bb None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:23.404707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8aa6fec5-5851-4d91-9596-9a529f6142bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:23.504547 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:23.521636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1339/2663] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:28:23.536675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:23.538677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:23.586208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:23.634159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f821fb-732f-4229-813d-aad0737f2e4e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:23.637686 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91895ae1-9a31-42bc-86ed-4fd74af2f424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:23.638342 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91895ae1-9a31-42bc-86ed-4fd74af2f424 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:23.638624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91895ae1-9a31-42bc-86ed-4fd74af2f424 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:23.638898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91895ae1-9a31-42bc-86ed-4fd74af2f424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:23.648867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91895ae1-9a31-42bc-86ed-4fd74af2f424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:23.650372 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91895ae1-9a31-42bc-86ed-4fd74af2f424 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:23.663584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1325/2664] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:23.760705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6fec5-5851-4d91-9596-9a529f6142bb None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:23.762571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8aa6fec5-5851-4d91-9596-9a529f6142bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:23.794347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6fec5-5851-4d91-9596-9a529f6142bb None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:23.802209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:23.803065 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:23.803371 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:23.803752 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] PATH_INFO: `/v3/users/86f2cda914b74f979746f4fb370eaae6` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:23.805213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f2cda914b74f979746f4fb370eaae6)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:23.807606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:23.872933 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f2cda914b74f979746f4fb370eaae6'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:23.882788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f2cda914b74f979746f4fb370eaae6'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:23.883721 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f2cda914b74f979746f4fb370eaae6'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:23.898738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f2cda914b74f979746f4fb370eaae6'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:23.899618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f2cda914b74f979746f4fb370eaae6'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:23.916113 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1340/2665] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:23 2022] DELETE /identity/v3/users/86f2cda914b74f979746f4fb370eaae6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:23.990692 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91895ae1-9a31-42bc-86ed-4fd74af2f424 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:23.992324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-91895ae1-9a31-42bc-86ed-4fd74af2f424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:24.010673 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91895ae1-9a31-42bc-86ed-4fd74af2f424 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:24.083266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:24.084457 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:24.084719 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:24.085175 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] PATH_INFO: `/v3/projects/754d6f6c545f4e70b08cbc2eaa2a6c13` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:24.086327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] RBAC: Authorizing `identity:delete_project(project_id=754d6f6c545f4e70b08cbc2eaa2a6c13)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:24.088449 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:24.112869 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:24.114408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:24.134295 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f56ed-8723-423f-9951-138136522b64 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:24.147399 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '754d6f6c545f4e70b08cbc2eaa2a6c13', 'request_id': 'req-35b25dfa-b0e4-4b79-a79d-621d4143449b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.155905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '754d6f6c545f4e70b08cbc2eaa2a6c13', 'request_id': 'req-35b25dfa-b0e4-4b79-a79d-621d4143449b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.163980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '754d6f6c545f4e70b08cbc2eaa2a6c13', 'request_id': 'req-35b25dfa-b0e4-4b79-a79d-621d4143449b'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.175054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] The token cache is being invalidate because project 754d6f6c545f4e70b08cbc2eaa2a6c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:24.175941 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.176297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.177906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1326/2666] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:23 2022] DELETE /identity/v3/projects/754d6f6c545f4e70b08cbc2eaa2a6c13 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:24.181968 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:24.183826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:24.198683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:24.199559 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:24.199845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:24.200165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] PATH_INFO: `/v3/projects/f92a857cef0a43d4907fbc667e30df6d` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:24.201458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f92a857cef0a43d4907fbc667e30df6d)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:24.203398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:24.254381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f92a857cef0a43d4907fbc667e30df6d', 'request_id': 'req-266f10fb-38ab-4d61-a563-6b0a2cb0864c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.254944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b25dfa-b0e4-4b79-a79d-621d4143449b admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:24.261244 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f92a857cef0a43d4907fbc667e30df6d', 'request_id': 'req-266f10fb-38ab-4d61-a563-6b0a2cb0864c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.263773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:24.269575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f92a857cef0a43d4907fbc667e30df6d', 'request_id': 'req-266f10fb-38ab-4d61-a563-6b0a2cb0864c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.279262 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:24.285477 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] The token cache is being invalidate because project f92a857cef0a43d4907fbc667e30df6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:24.286278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.287570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.290182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1341/2667] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:24 2022] DELETE /identity/v3/projects/f92a857cef0a43d4907fbc667e30df6d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:28:24.293781 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:24.295159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:24.327703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:24.328651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:24.329644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:24.329955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] PATH_INFO: `/v3/users/1cf6216ce2b446d3a5da725311e507e3` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:24.331304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cf6216ce2b446d3a5da725311e507e3)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:24.333613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:24.374364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f10fb-38ab-4d61-a563-6b0a2cb0864c admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:24.380519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:24.381435 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:24.381714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:24.381992 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] PATH_INFO: `/v3/users/1b657698fc114139b4e3655ca44291b6` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:24.383251 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b657698fc114139b4e3655ca44291b6)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:24.385338 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:24.385475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cf6216ce2b446d3a5da725311e507e3'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.386244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cf6216ce2b446d3a5da725311e507e3'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.397987 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cf6216ce2b446d3a5da725311e507e3'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.413314 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cf6216ce2b446d3a5da725311e507e3'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.414085 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cf6216ce2b446d3a5da725311e507e3'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.428250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1327/2668] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:24 2022] DELETE /identity/v3/users/1cf6216ce2b446d3a5da725311e507e3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:28:24.443369 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b657698fc114139b4e3655ca44291b6'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.453598 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b657698fc114139b4e3655ca44291b6'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.454969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b657698fc114139b4e3655ca44291b6'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.471250 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b657698fc114139b4e3655ca44291b6'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.472074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b657698fc114139b4e3655ca44291b6'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.485505 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1342/2669] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:24 2022] DELETE /identity/v3/users/1b657698fc114139b4e3655ca44291b6 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:24.728592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:24.730324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:24.778202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:24.779673 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:24.849824 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52392b16-8ff8-42a0-8878-d8e67ab81e21 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:24.854849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e86ed4-41e8-460d-9138-a2874e1bb829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:24.855744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e86ed4-41e8-460d-9138-a2874e1bb829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:24.856017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e86ed4-41e8-460d-9138-a2874e1bb829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:24.856331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e86ed4-41e8-460d-9138-a2874e1bb829 admin admin] PATH_INFO: `/v3/projects/e7c470fb14fb41878df38dde6131b126` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:24.857551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e86ed4-41e8-460d-9138-a2874e1bb829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c470fb14fb41878df38dde6131b126)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:24.859476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e86ed4-41e8-460d-9138-a2874e1bb829 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:24.861319 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868a101-5289-41bc-98d6-f2f7bf764bb8 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:24.868843 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:24.872990 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:24.873523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:24.873942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77 admin admin] PATH_INFO: `/v3/projects/f42e869a309241e3bf0fb6f348fef0e9` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:24.875445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f42e869a309241e3bf0fb6f348fef0e9)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:24.878191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:24.901352 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:24.902343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:24.918069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-29e86ed4-41e8-460d-9138-a2874e1bb829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c470fb14fb41878df38dde6131b126', 'request_id': 'req-29e86ed4-41e8-460d-9138-a2874e1bb829'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.928305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-29e86ed4-41e8-460d-9138-a2874e1bb829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c470fb14fb41878df38dde6131b126', 'request_id': 'req-29e86ed4-41e8-460d-9138-a2874e1bb829'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.932406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42e869a309241e3bf0fb6f348fef0e9', 'request_id': 'req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.935347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-29e86ed4-41e8-460d-9138-a2874e1bb829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c470fb14fb41878df38dde6131b126', 'request_id': 'req-29e86ed4-41e8-460d-9138-a2874e1bb829'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.939828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42e869a309241e3bf0fb6f348fef0e9', 'request_id': 'req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.946415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42e869a309241e3bf0fb6f348fef0e9', 'request_id': 'req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.948287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-29e86ed4-41e8-460d-9138-a2874e1bb829 admin admin] The token cache is being invalidate because project e7c470fb14fb41878df38dde6131b126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:24.948706 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-29e86ed4-41e8-460d-9138-a2874e1bb829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.949656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-29e86ed4-41e8-460d-9138-a2874e1bb829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.951403 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1328/2670] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:24 2022] DELETE /identity/v3/projects/e7c470fb14fb41878df38dde6131b126 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:24.961116 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77 admin admin] The token cache is being invalidate because project f42e869a309241e3bf0fb6f348fef0e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:24.961538 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.962173 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-a243dff1-fc21-4f1e-b7f7-0ad59ee7eb77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:24.964644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1343/2671] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:24 2022] DELETE /identity/v3/projects/f42e869a309241e3bf0fb6f348fef0e9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:28:24.970097 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff64adb-2b14-465f-8db7-37b3a2bcf3c5 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:24.970434 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff64adb-2b14-465f-8db7-37b3a2bcf3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:24.970746 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff64adb-2b14-465f-8db7-37b3a2bcf3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:24.993760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-aff64adb-2b14-465f-8db7-37b3a2bcf3c5 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:24.995205 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-aff64adb-2b14-465f-8db7-37b3a2bcf3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:25.068596 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1329/2672] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:28:25.076919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380974b-de25-4dbb-bee7-928f220fc436 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.077212 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380974b-de25-4dbb-bee7-928f220fc436 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.077498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380974b-de25-4dbb-bee7-928f220fc436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.097083 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-b380974b-de25-4dbb-bee7-928f220fc436 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:25.098351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b380974b-de25-4dbb-bee7-928f220fc436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:25.107311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1344/2673] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:25.112724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff64adb-2b14-465f-8db7-37b3a2bcf3c5 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:25.122886 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff64adb-2b14-465f-8db7-37b3a2bcf3c5 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:25.128758 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fd999-5386-4df6-a87d-99fb31eb3242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:25.129458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fd999-5386-4df6-a87d-99fb31eb3242 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.129743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fd999-5386-4df6-a87d-99fb31eb3242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.130017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fd999-5386-4df6-a87d-99fb31eb3242 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.131005 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fd999-5386-4df6-a87d-99fb31eb3242 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:28:25.131342 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fd999-5386-4df6-a87d-99fb31eb3242 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:25.133064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fd999-5386-4df6-a87d-99fb31eb3242 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:25.148415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1330/2674] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:28:25.152935 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380974b-de25-4dbb-bee7-928f220fc436 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:25.162951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380974b-de25-4dbb-bee7-928f220fc436 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:25.168410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b855ed3-6ecb-4780-a4a4-9b9cb4fd91f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:25.168837 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b855ed3-6ecb-4780-a4a4-9b9cb4fd91f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.169127 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b855ed3-6ecb-4780-a4a4-9b9cb4fd91f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.169410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b855ed3-6ecb-4780-a4a4-9b9cb4fd91f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.170124 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b855ed3-6ecb-4780-a4a4-9b9cb4fd91f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:25.172118 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b855ed3-6ecb-4780-a4a4-9b9cb4fd91f6 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:25.204232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1345/2675] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:28:25 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:25.208503 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fd999-5386-4df6-a87d-99fb31eb3242 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:25.218640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fd999-5386-4df6-a87d-99fb31eb3242 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:25.277671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674f59e-65b4-4223-8529-7b4772f18197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:25.278347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674f59e-65b4-4223-8529-7b4772f18197 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.278636 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674f59e-65b4-4223-8529-7b4772f18197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.278913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674f59e-65b4-4223-8529-7b4772f18197 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.279604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674f59e-65b4-4223-8529-7b4772f18197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:25.281568 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674f59e-65b4-4223-8529-7b4772f18197 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:25.303671 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1331/2676] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:28:25 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:25.308177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b855ed3-6ecb-4780-a4a4-9b9cb4fd91f6 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:25.318732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b855ed3-6ecb-4780-a4a4-9b9cb4fd91f6 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:25.323341 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f27ba-c174-4e1f-bd1c-2e59d72e56ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:25.324005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f27ba-c174-4e1f-bd1c-2e59d72e56ae admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.324320 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f27ba-c174-4e1f-bd1c-2e59d72e56ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.324597 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f27ba-c174-4e1f-bd1c-2e59d72e56ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.325323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f27ba-c174-4e1f-bd1c-2e59d72e56ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:25.326922 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f27ba-c174-4e1f-bd1c-2e59d72e56ae admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:25.337942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1346/2677] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:25.342072 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674f59e-65b4-4223-8529-7b4772f18197 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:25.353744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674f59e-65b4-4223-8529-7b4772f18197 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:25.358469 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbb68a-fb95-434f-b701-c0bcd8a82c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:25.359139 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbb68a-fb95-434f-b701-c0bcd8a82c30 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.359425 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbb68a-fb95-434f-b701-c0bcd8a82c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.359703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbb68a-fb95-434f-b701-c0bcd8a82c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.360523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbb68a-fb95-434f-b701-c0bcd8a82c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:25.362126 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbb68a-fb95-434f-b701-c0bcd8a82c30 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:25.372266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1332/2678] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:25.377042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f27ba-c174-4e1f-bd1c-2e59d72e56ae admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:25.387272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f27ba-c174-4e1f-bd1c-2e59d72e56ae admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:25.392696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d2f68-e277-41ec-87a7-af710cc0dfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:25.393602 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d2f68-e277-41ec-87a7-af710cc0dfe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.393937 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d2f68-e277-41ec-87a7-af710cc0dfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.394294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d2f68-e277-41ec-87a7-af710cc0dfe4 admin admin] PATH_INFO: `/v3/projects/436f67d06c2b41eb8162e48171a62d90/users/dd1facd88e064a3b8bbeac29922b3ba4/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.410923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d2f68-e277-41ec-87a7-af710cc0dfe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=436f67d06c2b41eb8162e48171a62d90, user_id=dd1facd88e064a3b8bbeac29922b3ba4, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:25.413927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d2f68-e277-41ec-87a7-af710cc0dfe4 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:25.427760 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1347/2679] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:25 2022] PUT /identity/v3/projects/436f67d06c2b41eb8162e48171a62d90/users/dd1facd88e064a3b8bbeac29922b3ba4/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:25.438574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf612ef-1400-428a-9d12-1fdd8e3a89ec None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.438871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf612ef-1400-428a-9d12-1fdd8e3a89ec None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.439150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf612ef-1400-428a-9d12-1fdd8e3a89ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.466901 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ccf612ef-1400-428a-9d12-1fdd8e3a89ec None None] MFA Rules not processed for user `dd1facd88e064a3b8bbeac29922b3ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:25.468301 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ccf612ef-1400-428a-9d12-1fdd8e3a89ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:25.482677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:25.537316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1333/2680] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:25.541786 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d2f68-e277-41ec-87a7-af710cc0dfe4 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:25.603924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d2f68-e277-41ec-87a7-af710cc0dfe4 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:25.608868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0269938-3fd6-412b-a5c3-9f87ee91a147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:25.609539 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0269938-3fd6-412b-a5c3-9f87ee91a147 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.609829 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0269938-3fd6-412b-a5c3-9f87ee91a147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.610634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0269938-3fd6-412b-a5c3-9f87ee91a147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.611408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0269938-3fd6-412b-a5c3-9f87ee91a147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:25.613303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0269938-3fd6-412b-a5c3-9f87ee91a147 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:25.639646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1348/2681] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:28:25 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:25.644376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf612ef-1400-428a-9d12-1fdd8e3a89ec None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:25.655415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf612ef-1400-428a-9d12-1fdd8e3a89ec None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:25.715141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d466717-168e-48e1-bfdb-4a774c517a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:25.715818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d466717-168e-48e1-bfdb-4a774c517a7d admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.716140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d466717-168e-48e1-bfdb-4a774c517a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.716417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d466717-168e-48e1-bfdb-4a774c517a7d admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.717121 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d466717-168e-48e1-bfdb-4a774c517a7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:25.719040 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d466717-168e-48e1-bfdb-4a774c517a7d admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:25.739883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1334/2682] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:28:25 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:25.744739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0269938-3fd6-412b-a5c3-9f87ee91a147 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:25.756192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0269938-3fd6-412b-a5c3-9f87ee91a147 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:25.761040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02577189-4cb9-4ab2-8064-04cad6108392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:25.761711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02577189-4cb9-4ab2-8064-04cad6108392 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.761994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02577189-4cb9-4ab2-8064-04cad6108392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.762272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02577189-4cb9-4ab2-8064-04cad6108392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.763005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02577189-4cb9-4ab2-8064-04cad6108392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:25.764641 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02577189-4cb9-4ab2-8064-04cad6108392 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:25.775867 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1349/2683] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:25.780058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d466717-168e-48e1-bfdb-4a774c517a7d admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:25.791039 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d466717-168e-48e1-bfdb-4a774c517a7d admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:25.795919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998d22d-1669-4b55-9609-97395d1631be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:25.796847 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998d22d-1669-4b55-9609-97395d1631be admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.797162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998d22d-1669-4b55-9609-97395d1631be admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.797958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998d22d-1669-4b55-9609-97395d1631be admin admin] PATH_INFO: `/v3/projects/786e2ac1bb0e458aa05396e074b6c69d/users/918a8190a90c4bd8a9d457c7f8080992/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.814989 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998d22d-1669-4b55-9609-97395d1631be admin admin] RBAC: Authorizing `identity:create_grant(project_id=786e2ac1bb0e458aa05396e074b6c69d, user_id=918a8190a90c4bd8a9d457c7f8080992, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:25.817705 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998d22d-1669-4b55-9609-97395d1631be admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:25.831141 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1335/2684] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:25 2022] PUT /identity/v3/projects/786e2ac1bb0e458aa05396e074b6c69d/users/918a8190a90c4bd8a9d457c7f8080992/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:25.835145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02577189-4cb9-4ab2-8064-04cad6108392 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:25.845707 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02577189-4cb9-4ab2-8064-04cad6108392 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:25.861732 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:25.907522 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746b8f6-c9dd-498f-8477-6356091c3627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:25.908234 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746b8f6-c9dd-498f-8477-6356091c3627 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.908526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746b8f6-c9dd-498f-8477-6356091c3627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.908806 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746b8f6-c9dd-498f-8477-6356091c3627 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.909541 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746b8f6-c9dd-498f-8477-6356091c3627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:25.911152 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746b8f6-c9dd-498f-8477-6356091c3627 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:25.921573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1350/2685] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:25.926731 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998d22d-1669-4b55-9609-97395d1631be admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:25.936945 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998d22d-1669-4b55-9609-97395d1631be admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:25.942086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd0b28-bbd1-4903-aea9-3e68367e7fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:25.942971 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd0b28-bbd1-4903-aea9-3e68367e7fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.943259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd0b28-bbd1-4903-aea9-3e68367e7fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.943534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd0b28-bbd1-4903-aea9-3e68367e7fbb admin admin] PATH_INFO: `/v3/projects/786e2ac1bb0e458aa05396e074b6c69d/users/918a8190a90c4bd8a9d457c7f8080992/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.945554 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd0b28-bbd1-4903-aea9-3e68367e7fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=786e2ac1bb0e458aa05396e074b6c69d, user_id=918a8190a90c4bd8a9d457c7f8080992, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:25.948339 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd0b28-bbd1-4903-aea9-3e68367e7fbb admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:25.959019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1336/2686] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:25 2022] PUT /identity/v3/projects/786e2ac1bb0e458aa05396e074b6c69d/users/918a8190a90c4bd8a9d457c7f8080992/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:25.970232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e019d-a11d-433e-a788-a62deda76a58 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:25.970532 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e019d-a11d-433e-a788-a62deda76a58 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:25.970887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e019d-a11d-433e-a788-a62deda76a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:25.999374 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-f26e019d-a11d-433e-a788-a62deda76a58 None None] MFA Rules not processed for user `918a8190a90c4bd8a9d457c7f8080992`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:26.000715 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f26e019d-a11d-433e-a788-a62deda76a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:26.071071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1351/2687] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:26.083195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd0b28-bbd1-4903-aea9-3e68367e7fbb admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:26.084962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-89cd0b28-bbd1-4903-aea9-3e68367e7fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:26.142451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd0b28-bbd1-4903-aea9-3e68367e7fbb admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:26.146087 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e0c80-0928-46b6-a852-60effd01489c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:26.146744 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e0c80-0928-46b6-a852-60effd01489c service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:26.147028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e0c80-0928-46b6-a852-60effd01489c service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:26.147306 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e0c80-0928-46b6-a852-60effd01489c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:26.158466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e0c80-0928-46b6-a852-60effd01489c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:26.159948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e0c80-0928-46b6-a852-60effd01489c service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:26.170223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1337/2688] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:26.495126 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e019d-a11d-433e-a788-a62deda76a58 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:26.504552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e019d-a11d-433e-a788-a62deda76a58 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:26.509408 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70921463-0f4d-4ece-824a-b3cda5cb6299 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918a8190a90c4bd8a9d457c7f8080992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '786e2ac1bb0e458aa05396e074b6c69d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:26.510106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70921463-0f4d-4ece-824a-b3cda5cb6299 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:26.510398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70921463-0f4d-4ece-824a-b3cda5cb6299 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:26.510679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70921463-0f4d-4ece-824a-b3cda5cb6299 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:26.511413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70921463-0f4d-4ece-824a-b3cda5cb6299 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:28:26.511725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70921463-0f4d-4ece-824a-b3cda5cb6299 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:26.513421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70921463-0f4d-4ece-824a-b3cda5cb6299 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:26.525988 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1352/2689] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:28:26.530447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e0c80-0928-46b6-a852-60effd01489c service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:26.540806 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e0c80-0928-46b6-a852-60effd01489c service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:26.545567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19054462-7c01-461d-a6ac-0efcc53b63ee tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918a8190a90c4bd8a9d457c7f8080992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '786e2ac1bb0e458aa05396e074b6c69d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:26.546236 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19054462-7c01-461d-a6ac-0efcc53b63ee tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:26.546535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19054462-7c01-461d-a6ac-0efcc53b63ee tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:26.546821 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19054462-7c01-461d-a6ac-0efcc53b63ee tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:26.547538 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19054462-7c01-461d-a6ac-0efcc53b63ee tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:26.549498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19054462-7c01-461d-a6ac-0efcc53b63ee tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:26.558448 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:26.580598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1338/2690] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:28:26 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:26.582666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70921463-0f4d-4ece-824a-b3cda5cb6299 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:26.592995 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70921463-0f4d-4ece-824a-b3cda5cb6299 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:26.649374 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89714d-827f-42bd-8b13-5aeb979c0425 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918a8190a90c4bd8a9d457c7f8080992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '786e2ac1bb0e458aa05396e074b6c69d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:26.650048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89714d-827f-42bd-8b13-5aeb979c0425 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:26.650348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89714d-827f-42bd-8b13-5aeb979c0425 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:26.650633 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89714d-827f-42bd-8b13-5aeb979c0425 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:26.651373 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89714d-827f-42bd-8b13-5aeb979c0425 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:28:26.651687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89714d-827f-42bd-8b13-5aeb979c0425 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:26.653367 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89714d-827f-42bd-8b13-5aeb979c0425 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:26.666267 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1353/2691] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 07:28:26.703898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19054462-7c01-461d-a6ac-0efcc53b63ee tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:26.714422 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19054462-7c01-461d-a6ac-0efcc53b63ee tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:26.719481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34018bbc-8fca-4653-b0e9-7701ea1be6db tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918a8190a90c4bd8a9d457c7f8080992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '786e2ac1bb0e458aa05396e074b6c69d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:26.720173 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34018bbc-8fca-4653-b0e9-7701ea1be6db tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:26.720466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34018bbc-8fca-4653-b0e9-7701ea1be6db tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:26.720749 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34018bbc-8fca-4653-b0e9-7701ea1be6db tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] PATH_INFO: `/v3/projects/4cd35262f063494f8556418d0f648176` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:26.722016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34018bbc-8fca-4653-b0e9-7701ea1be6db tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4cd35262f063494f8556418d0f648176)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:26.723860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34018bbc-8fca-4653-b0e9-7701ea1be6db tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:26.772024 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-34018bbc-8fca-4653-b0e9-7701ea1be6db tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd35262f063494f8556418d0f648176', 'request_id': 'req-34018bbc-8fca-4653-b0e9-7701ea1be6db'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:26.781142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-34018bbc-8fca-4653-b0e9-7701ea1be6db tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd35262f063494f8556418d0f648176', 'request_id': 'req-34018bbc-8fca-4653-b0e9-7701ea1be6db'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:26.788292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-34018bbc-8fca-4653-b0e9-7701ea1be6db tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd35262f063494f8556418d0f648176', 'request_id': 'req-34018bbc-8fca-4653-b0e9-7701ea1be6db'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:26.799214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-34018bbc-8fca-4653-b0e9-7701ea1be6db tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] The token cache is being invalidate because project 4cd35262f063494f8556418d0f648176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:26.799502 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-34018bbc-8fca-4653-b0e9-7701ea1be6db tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:26.800195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-34018bbc-8fca-4653-b0e9-7701ea1be6db tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:26.802331 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1339/2692] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:26 2022] DELETE /identity/v3/projects/4cd35262f063494f8556418d0f648176 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:26.998083 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89714d-827f-42bd-8b13-5aeb979c0425 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:26.999822 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-bf89714d-827f-42bd-8b13-5aeb979c0425 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:27.022570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=100 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:27.067788 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89714d-827f-42bd-8b13-5aeb979c0425 tempest-VolumeQuotaClassesTest-1440679026 tempest-VolumeQuotaClassesTest-1440679026-project-admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:27.125017 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:27.125924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:27.126219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:27.126500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] PATH_INFO: `/v3/users/dd1facd88e064a3b8bbeac29922b3ba4` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:27.127698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd1facd88e064a3b8bbeac29922b3ba4)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:27.129674 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:27.179622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd1facd88e064a3b8bbeac29922b3ba4'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.187636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd1facd88e064a3b8bbeac29922b3ba4'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.188406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd1facd88e064a3b8bbeac29922b3ba4'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.203299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd1facd88e064a3b8bbeac29922b3ba4'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.204147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd1facd88e064a3b8bbeac29922b3ba4'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.217638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1354/2693] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:26 2022] DELETE /identity/v3/users/dd1facd88e064a3b8bbeac29922b3ba4 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:27.222624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ad3c7-3f11-4446-a42f-cfe234ed614d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:27.222919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ad3c7-3f11-4446-a42f-cfe234ed614d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:27.223204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ad3c7-3f11-4446-a42f-cfe234ed614d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:27.241520 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-534ad3c7-3f11-4446-a42f-cfe234ed614d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:27.242946 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-534ad3c7-3f11-4446-a42f-cfe234ed614d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:27.304735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1340/2694] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:27.314723 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:27.316342 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:27.376828 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:27.384056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263efeb-9632-47ca-9b84-a78ebe108ace admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:27.388364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf519199-8e29-46a6-9179-2c9a96cf1e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:27.389149 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf519199-8e29-46a6-9179-2c9a96cf1e04 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:27.389564 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf519199-8e29-46a6-9179-2c9a96cf1e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:27.389942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf519199-8e29-46a6-9179-2c9a96cf1e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:27.400458 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf519199-8e29-46a6-9179-2c9a96cf1e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:27.401921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf519199-8e29-46a6-9179-2c9a96cf1e04 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:27.412802 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1355/2695] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:27.685365 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534ad3c7-3f11-4446-a42f-cfe234ed614d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:27.687015 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-534ad3c7-3f11-4446-a42f-cfe234ed614d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:27.699848 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534ad3c7-3f11-4446-a42f-cfe234ed614d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:27.705312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:27.706212 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:27.706642 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:27.706929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] PATH_INFO: `/v3/projects/436f67d06c2b41eb8162e48171a62d90` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:27.708183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=436f67d06c2b41eb8162e48171a62d90)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:27.710116 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:27.762361 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '436f67d06c2b41eb8162e48171a62d90', 'request_id': 'req-238e5fc4-89c0-40cf-81a1-48efe4cec366'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.770707 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '436f67d06c2b41eb8162e48171a62d90', 'request_id': 'req-238e5fc4-89c0-40cf-81a1-48efe4cec366'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.777597 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '436f67d06c2b41eb8162e48171a62d90', 'request_id': 'req-238e5fc4-89c0-40cf-81a1-48efe4cec366'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.782432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=14 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:27.789989 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] The token cache is being invalidate because project 436f67d06c2b41eb8162e48171a62d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:27.790292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.791321 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.792957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1341/2696] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:27 2022] DELETE /identity/v3/projects/436f67d06c2b41eb8162e48171a62d90 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:27.796947 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf519199-8e29-46a6-9179-2c9a96cf1e04 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:27.807259 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf519199-8e29-46a6-9179-2c9a96cf1e04 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:27.864240 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f369ab-fcb5-466a-8970-c4dd19731345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:27.865038 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f369ab-fcb5-466a-8970-c4dd19731345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:27.865320 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f369ab-fcb5-466a-8970-c4dd19731345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:27.865596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f369ab-fcb5-466a-8970-c4dd19731345 admin admin] PATH_INFO: `/v3/users/918a8190a90c4bd8a9d457c7f8080992` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:27.866821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f369ab-fcb5-466a-8970-c4dd19731345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=918a8190a90c4bd8a9d457c7f8080992)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:27.868830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f369ab-fcb5-466a-8970-c4dd19731345 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:27.921134 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d5f369ab-fcb5-466a-8970-c4dd19731345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '918a8190a90c4bd8a9d457c7f8080992'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.929661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d5f369ab-fcb5-466a-8970-c4dd19731345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '918a8190a90c4bd8a9d457c7f8080992'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.930289 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d5f369ab-fcb5-466a-8970-c4dd19731345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '918a8190a90c4bd8a9d457c7f8080992'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.943091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d5f369ab-fcb5-466a-8970-c4dd19731345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '918a8190a90c4bd8a9d457c7f8080992'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.943777 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d5f369ab-fcb5-466a-8970-c4dd19731345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '918a8190a90c4bd8a9d457c7f8080992'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:27.957635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1356/2697] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:27 2022] DELETE /identity/v3/users/918a8190a90c4bd8a9d457c7f8080992 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:28.247391 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:28.248964 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:28.262917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e5fc4-89c0-40cf-81a1-48efe4cec366 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:28.326923 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97c985-e544-4913-84a6-f044376e65a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:28.327793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97c985-e544-4913-84a6-f044376e65a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:28.328092 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97c985-e544-4913-84a6-f044376e65a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:28.328386 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97c985-e544-4913-84a6-f044376e65a6 admin admin] PATH_INFO: `/v3/projects/786e2ac1bb0e458aa05396e074b6c69d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:28.329602 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97c985-e544-4913-84a6-f044376e65a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=786e2ac1bb0e458aa05396e074b6c69d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:28.331551 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97c985-e544-4913-84a6-f044376e65a6 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:28.382969 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1e97c985-e544-4913-84a6-f044376e65a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '786e2ac1bb0e458aa05396e074b6c69d', 'request_id': 'req-1e97c985-e544-4913-84a6-f044376e65a6'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:28.391892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1e97c985-e544-4913-84a6-f044376e65a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '786e2ac1bb0e458aa05396e074b6c69d', 'request_id': 'req-1e97c985-e544-4913-84a6-f044376e65a6'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:28.398785 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1e97c985-e544-4913-84a6-f044376e65a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '786e2ac1bb0e458aa05396e074b6c69d', 'request_id': 'req-1e97c985-e544-4913-84a6-f044376e65a6'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:28.410166 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1e97c985-e544-4913-84a6-f044376e65a6 admin admin] The token cache is being invalidate because project 786e2ac1bb0e458aa05396e074b6c69d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:28.410442 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1e97c985-e544-4913-84a6-f044376e65a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:28.411027 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1e97c985-e544-4913-84a6-f044376e65a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:28.412631 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1342/2698] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:28 2022] DELETE /identity/v3/projects/786e2ac1bb0e458aa05396e074b6c69d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:28.417679 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715bb53-8500-4c58-9bfa-6a811f677513 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:28.417975 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715bb53-8500-4c58-9bfa-6a811f677513 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:28.418258 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715bb53-8500-4c58-9bfa-6a811f677513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:28.437051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-a715bb53-8500-4c58-9bfa-6a811f677513 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:28.438388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-a715bb53-8500-4c58-9bfa-6a811f677513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:28.499303 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1357/2699] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:28.507384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa0f53-c53a-49c6-b9a0-2049f5543a10 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:28.507676 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa0f53-c53a-49c6-b9a0-2049f5543a10 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:28.507963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa0f53-c53a-49c6-b9a0-2049f5543a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:28.527511 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-43aa0f53-c53a-49c6-b9a0-2049f5543a10 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:28.528760 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-43aa0f53-c53a-49c6-b9a0-2049f5543a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:28.537230 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1343/2700] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:28.541575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715bb53-8500-4c58-9bfa-6a811f677513 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:28.553192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715bb53-8500-4c58-9bfa-6a811f677513 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:28.558787 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65065a-db38-4532-8466-85a1fc7b9381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:28.559570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65065a-db38-4532-8466-85a1fc7b9381 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:28.559860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65065a-db38-4532-8466-85a1fc7b9381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:28.560172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65065a-db38-4532-8466-85a1fc7b9381 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:28.560902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65065a-db38-4532-8466-85a1fc7b9381 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:28:28.561212 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65065a-db38-4532-8466-85a1fc7b9381 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:28.562814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65065a-db38-4532-8466-85a1fc7b9381 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:28.565405 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:28.576311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1358/2701] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:28:28.580433 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa0f53-c53a-49c6-b9a0-2049f5543a10 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:28.589926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa0f53-c53a-49c6-b9a0-2049f5543a10 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:28.595168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ba35b-bfc3-474e-b594-31d3cb2e6d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:28.595827 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ba35b-bfc3-474e-b594-31d3cb2e6d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:28.596146 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ba35b-bfc3-474e-b594-31d3cb2e6d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:28.596430 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ba35b-bfc3-474e-b594-31d3cb2e6d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:28.597119 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ba35b-bfc3-474e-b594-31d3cb2e6d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:28.599036 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ba35b-bfc3-474e-b594-31d3cb2e6d8b admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:28.629784 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1344/2702] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:28:28 2022] POST /identity/v3/projects => generated 397 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:28.633896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65065a-db38-4532-8466-85a1fc7b9381 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:28.644029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65065a-db38-4532-8466-85a1fc7b9381 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:28.704021 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b346137-6092-4074-8095-a44f9fe4fa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:28.704728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b346137-6092-4074-8095-a44f9fe4fa5d admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:28.705011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b346137-6092-4074-8095-a44f9fe4fa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:28.705291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b346137-6092-4074-8095-a44f9fe4fa5d admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:28.705992 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b346137-6092-4074-8095-a44f9fe4fa5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:28.707923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b346137-6092-4074-8095-a44f9fe4fa5d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:28.729844 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1359/2703] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:28:28 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:28.733954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ba35b-bfc3-474e-b594-31d3cb2e6d8b admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:28.745120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ba35b-bfc3-474e-b594-31d3cb2e6d8b admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:28.749898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03885e98-5138-4044-9e28-554de36460e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:28.750604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03885e98-5138-4044-9e28-554de36460e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:28.750888 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03885e98-5138-4044-9e28-554de36460e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:28.751168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03885e98-5138-4044-9e28-554de36460e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:28.752030 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03885e98-5138-4044-9e28-554de36460e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:28.753662 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03885e98-5138-4044-9e28-554de36460e2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:28.764292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1345/2704] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:28.769077 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b346137-6092-4074-8095-a44f9fe4fa5d admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:28.779273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b346137-6092-4074-8095-a44f9fe4fa5d admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:28.784836 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b55d7f-80d4-4b4c-86d7-a7e5cdfc9fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:28.785511 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b55d7f-80d4-4b4c-86d7-a7e5cdfc9fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:28.785800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b55d7f-80d4-4b4c-86d7-a7e5cdfc9fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:28.786085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b55d7f-80d4-4b4c-86d7-a7e5cdfc9fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:28.786815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b55d7f-80d4-4b4c-86d7-a7e5cdfc9fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:28.788463 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b55d7f-80d4-4b4c-86d7-a7e5cdfc9fe5 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:28.799221 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1360/2705] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:28.803919 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03885e98-5138-4044-9e28-554de36460e2 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:28.814133 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03885e98-5138-4044-9e28-554de36460e2 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:28.819005 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627e9f0-3e1a-4077-ae57-2da91801f5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:28.819890 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627e9f0-3e1a-4077-ae57-2da91801f5a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:28.820217 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627e9f0-3e1a-4077-ae57-2da91801f5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:28.820500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627e9f0-3e1a-4077-ae57-2da91801f5a3 admin admin] PATH_INFO: `/v3/projects/78060f4ba6264f4abafaf89ef3b54ac4/users/85f2f8dc9b2a432c8490ab44ba3d757c/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:28.836797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627e9f0-3e1a-4077-ae57-2da91801f5a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78060f4ba6264f4abafaf89ef3b54ac4, user_id=85f2f8dc9b2a432c8490ab44ba3d757c, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:28.839490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627e9f0-3e1a-4077-ae57-2da91801f5a3 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:28.853613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1346/2706] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:28 2022] PUT /identity/v3/projects/78060f4ba6264f4abafaf89ef3b54ac4/users/85f2f8dc9b2a432c8490ab44ba3d757c/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:28.864164 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1132143-8be3-4ec7-8fb3-922d2bff22e0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:28.864516 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1132143-8be3-4ec7-8fb3-922d2bff22e0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:28.864803 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1132143-8be3-4ec7-8fb3-922d2bff22e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:28.893211 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-c1132143-8be3-4ec7-8fb3-922d2bff22e0 None None] MFA Rules not processed for user `85f2f8dc9b2a432c8490ab44ba3d757c`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:28.894700 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c1132143-8be3-4ec7-8fb3-922d2bff22e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:28.965209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1361/2707] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:28:28.970376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627e9f0-3e1a-4077-ae57-2da91801f5a3 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:28.995497 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:29.033802 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627e9f0-3e1a-4077-ae57-2da91801f5a3 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:29.041392 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ff7d5-21d9-4cbd-9bf4-6e4e96829d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:29.042264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ff7d5-21d9-4cbd-9bf4-6e4e96829d78 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:29.042553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ff7d5-21d9-4cbd-9bf4-6e4e96829d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:29.042835 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ff7d5-21d9-4cbd-9bf4-6e4e96829d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:29.043540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ff7d5-21d9-4cbd-9bf4-6e4e96829d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:29.045655 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ff7d5-21d9-4cbd-9bf4-6e4e96829d78 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:29.073207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1347/2708] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:28:28 2022] POST /identity/v3/projects => generated 399 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:29.077313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1132143-8be3-4ec7-8fb3-922d2bff22e0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:29.089492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1132143-8be3-4ec7-8fb3-922d2bff22e0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:29.147578 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df370193-1b9a-4804-8e17-de37324a7c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:29.148294 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df370193-1b9a-4804-8e17-de37324a7c80 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:29.148591 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df370193-1b9a-4804-8e17-de37324a7c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:29.150993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df370193-1b9a-4804-8e17-de37324a7c80 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:29.150993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df370193-1b9a-4804-8e17-de37324a7c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:29.151486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df370193-1b9a-4804-8e17-de37324a7c80 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:29.160476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:29.176750 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1362/2709] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:28:29 2022] POST /identity/v3/users => generated 306 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:29.178438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ff7d5-21d9-4cbd-9bf4-6e4e96829d78 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:29.189557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ff7d5-21d9-4cbd-9bf4-6e4e96829d78 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:29.194284 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778a6b0-c380-4b64-bc7d-386200831973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:29.194956 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778a6b0-c380-4b64-bc7d-386200831973 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:29.195239 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778a6b0-c380-4b64-bc7d-386200831973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:29.195518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778a6b0-c380-4b64-bc7d-386200831973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:29.196271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778a6b0-c380-4b64-bc7d-386200831973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:29.197872 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778a6b0-c380-4b64-bc7d-386200831973 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:29.210001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1348/2710] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:29.214573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df370193-1b9a-4804-8e17-de37324a7c80 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:29.227034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df370193-1b9a-4804-8e17-de37324a7c80 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:29.233036 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37b0cf-3fe8-49c9-8e60-39a0957f6ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:29.234091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37b0cf-3fe8-49c9-8e60-39a0957f6ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:29.234433 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37b0cf-3fe8-49c9-8e60-39a0957f6ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:29.234718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37b0cf-3fe8-49c9-8e60-39a0957f6ef8 admin admin] PATH_INFO: `/v3/projects/bb3c9c8d94cb4a0aa7616e912fb54dd8/users/7518973a44594fdeb62dbca12d95f019/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:29.253644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37b0cf-3fe8-49c9-8e60-39a0957f6ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3c9c8d94cb4a0aa7616e912fb54dd8, user_id=7518973a44594fdeb62dbca12d95f019, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:29.256349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37b0cf-3fe8-49c9-8e60-39a0957f6ef8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:29.270757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1363/2711] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:29 2022] PUT /identity/v3/projects/bb3c9c8d94cb4a0aa7616e912fb54dd8/users/7518973a44594fdeb62dbca12d95f019/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:29.276302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778a6b0-c380-4b64-bc7d-386200831973 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:29.286710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778a6b0-c380-4b64-bc7d-386200831973 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:29.349624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ae014-384e-4c7a-95bb-2880fc5f3cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:29.350298 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ae014-384e-4c7a-95bb-2880fc5f3cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:29.350580 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ae014-384e-4c7a-95bb-2880fc5f3cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:29.350861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ae014-384e-4c7a-95bb-2880fc5f3cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:29.351592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ae014-384e-4c7a-95bb-2880fc5f3cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:29.353225 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ae014-384e-4c7a-95bb-2880fc5f3cbf admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:29.363499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1349/2712] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:28:29.368535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37b0cf-3fe8-49c9-8e60-39a0957f6ef8 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:29.379706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37b0cf-3fe8-49c9-8e60-39a0957f6ef8 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:29.384647 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9299a-e845-4fba-82f9-4ee00f72ece1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:29.385562 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9299a-e845-4fba-82f9-4ee00f72ece1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:29.385907 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9299a-e845-4fba-82f9-4ee00f72ece1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:29.386159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9299a-e845-4fba-82f9-4ee00f72ece1 admin admin] PATH_INFO: `/v3/projects/bb3c9c8d94cb4a0aa7616e912fb54dd8/users/7518973a44594fdeb62dbca12d95f019/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:29.388286 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9299a-e845-4fba-82f9-4ee00f72ece1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3c9c8d94cb4a0aa7616e912fb54dd8, user_id=7518973a44594fdeb62dbca12d95f019, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:29.390996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9299a-e845-4fba-82f9-4ee00f72ece1 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:29.403432 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1364/2713] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:29 2022] PUT /identity/v3/projects/bb3c9c8d94cb4a0aa7616e912fb54dd8/users/7518973a44594fdeb62dbca12d95f019/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:29.415615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0dc96-ab54-44cf-b8f3-707ebb01330b None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:29.415949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0dc96-ab54-44cf-b8f3-707ebb01330b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:29.416698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0dc96-ab54-44cf-b8f3-707ebb01330b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:29.446821 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-31b0dc96-ab54-44cf-b8f3-707ebb01330b None None] MFA Rules not processed for user `7518973a44594fdeb62dbca12d95f019`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:29.448045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-31b0dc96-ab54-44cf-b8f3-707ebb01330b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:29.519988 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1350/2714] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:29 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:29.531413 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9299a-e845-4fba-82f9-4ee00f72ece1 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:29.533224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b0b9299a-e845-4fba-82f9-4ee00f72ece1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:29.596813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9299a-e845-4fba-82f9-4ee00f72ece1 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:29.600994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02529872-075b-4d6b-8663-919ca303104a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:29.601689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02529872-075b-4d6b-8663-919ca303104a service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:29.601976 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02529872-075b-4d6b-8663-919ca303104a service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:29.602252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02529872-075b-4d6b-8663-919ca303104a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:29.612986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02529872-075b-4d6b-8663-919ca303104a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:29.614491 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02529872-075b-4d6b-8663-919ca303104a service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:29.626306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1365/2715] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:29.706856 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0dc96-ab54-44cf-b8f3-707ebb01330b None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:29.717057 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0dc96-ab54-44cf-b8f3-707ebb01330b None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:29.720732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8918d-9a19-45f2-b85b-146fea8309b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:29.721391 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8918d-9a19-45f2-b85b-146fea8309b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:29.721677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8918d-9a19-45f2-b85b-146fea8309b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:29.721953 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8918d-9a19-45f2-b85b-146fea8309b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:29.736717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:29.786069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8918d-9a19-45f2-b85b-146fea8309b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:29.788148 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8918d-9a19-45f2-b85b-146fea8309b4 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:29.799193 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1351/2716] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:33.938145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02529872-075b-4d6b-8663-919ca303104a service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:34.006611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02529872-075b-4d6b-8663-919ca303104a service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:34.011177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:34.012080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:34.012379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:34.012657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] PATH_INFO: `/v3/users/85f2f8dc9b2a432c8490ab44ba3d757c` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:34.013863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f2f8dc9b2a432c8490ab44ba3d757c)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:34.015815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:34.066817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f2f8dc9b2a432c8490ab44ba3d757c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.074363 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f2f8dc9b2a432c8490ab44ba3d757c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.074986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f2f8dc9b2a432c8490ab44ba3d757c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.081142 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:34.090184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f2f8dc9b2a432c8490ab44ba3d757c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.091024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f2f8dc9b2a432c8490ab44ba3d757c'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.104962 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1366/2717] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:33 2022] DELETE /identity/v3/users/85f2f8dc9b2a432c8490ab44ba3d757c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:34.109801 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b73de-5989-4224-adf1-321735a90a7d None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:34.110312 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b73de-5989-4224-adf1-321735a90a7d None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:34.110441 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b73de-5989-4224-adf1-321735a90a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:34.128297 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-7a5b73de-5989-4224-adf1-321735a90a7d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:34.129827 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7a5b73de-5989-4224-adf1-321735a90a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:34.194016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1352/2718] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:34.203860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:34.205348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:34.266718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38006dce-d0eb-46cd-b4b8-4eeefedbec9d admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:34.270323 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e36a1-78bb-4fc4-a848-e066b60a5fa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:34.270978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e36a1-78bb-4fc4-a848-e066b60a5fa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:34.271261 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e36a1-78bb-4fc4-a848-e066b60a5fa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:34.271537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e36a1-78bb-4fc4-a848-e066b60a5fa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:34.281287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e36a1-78bb-4fc4-a848-e066b60a5fa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:34.282756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e36a1-78bb-4fc4-a848-e066b60a5fa8 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:34.292546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1367/2719] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:34.577833 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b73de-5989-4224-adf1-321735a90a7d None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:34.579960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7a5b73de-5989-4224-adf1-321735a90a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:34.593500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b73de-5989-4224-adf1-321735a90a7d None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:34.598898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:34.599603 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:34.599878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:34.600195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] PATH_INFO: `/v3/projects/78060f4ba6264f4abafaf89ef3b54ac4` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:34.601443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78060f4ba6264f4abafaf89ef3b54ac4)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:34.603376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:34.655181 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78060f4ba6264f4abafaf89ef3b54ac4', 'request_id': 'req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.663321 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78060f4ba6264f4abafaf89ef3b54ac4', 'request_id': 'req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.669590 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78060f4ba6264f4abafaf89ef3b54ac4', 'request_id': 'req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.681687 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] The token cache is being invalidate because project 78060f4ba6264f4abafaf89ef3b54ac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:34.681973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.682624 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.684220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1353/2720] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:34 2022] DELETE /identity/v3/projects/78060f4ba6264f4abafaf89ef3b54ac4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:34.689460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e36a1-78bb-4fc4-a848-e066b60a5fa8 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:34.701752 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e36a1-78bb-4fc4-a848-e066b60a5fa8 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:34.761741 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbb66c-41d2-43e5-9db5-3bb0ce7113da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:34.762660 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbb66c-41d2-43e5-9db5-3bb0ce7113da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:34.762927 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbb66c-41d2-43e5-9db5-3bb0ce7113da admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:34.763200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbb66c-41d2-43e5-9db5-3bb0ce7113da admin admin] PATH_INFO: `/v3/users/7518973a44594fdeb62dbca12d95f019` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:34.764469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbb66c-41d2-43e5-9db5-3bb0ce7113da admin admin] RBAC: Authorizing `identity:delete_user(user_id=7518973a44594fdeb62dbca12d95f019)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:34.766995 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbb66c-41d2-43e5-9db5-3bb0ce7113da admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:34.815943 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c1dbb66c-41d2-43e5-9db5-3bb0ce7113da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7518973a44594fdeb62dbca12d95f019'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.824414 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c1dbb66c-41d2-43e5-9db5-3bb0ce7113da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7518973a44594fdeb62dbca12d95f019'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.825038 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c1dbb66c-41d2-43e5-9db5-3bb0ce7113da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7518973a44594fdeb62dbca12d95f019'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.838454 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c1dbb66c-41d2-43e5-9db5-3bb0ce7113da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7518973a44594fdeb62dbca12d95f019'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.839178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c1dbb66c-41d2-43e5-9db5-3bb0ce7113da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7518973a44594fdeb62dbca12d95f019'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:34.848224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:34.853599 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1368/2721] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:34 2022] DELETE /identity/v3/users/7518973a44594fdeb62dbca12d95f019 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:35.133841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:35.135341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:35.151898 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd15d5-d668-4a61-bce0-4ca97d190ef2 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:35.193523 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:35.214043 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe5719-c999-4db6-a321-4691e2ca673e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:35.214913 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe5719-c999-4db6-a321-4691e2ca673e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:35.215198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe5719-c999-4db6-a321-4691e2ca673e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:35.215477 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe5719-c999-4db6-a321-4691e2ca673e admin admin] PATH_INFO: `/v3/projects/bb3c9c8d94cb4a0aa7616e912fb54dd8` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:35.216681 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe5719-c999-4db6-a321-4691e2ca673e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb3c9c8d94cb4a0aa7616e912fb54dd8)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:35.218593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe5719-c999-4db6-a321-4691e2ca673e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:35.265849 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1efe5719-c999-4db6-a321-4691e2ca673e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb3c9c8d94cb4a0aa7616e912fb54dd8', 'request_id': 'req-1efe5719-c999-4db6-a321-4691e2ca673e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:35.274396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1efe5719-c999-4db6-a321-4691e2ca673e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb3c9c8d94cb4a0aa7616e912fb54dd8', 'request_id': 'req-1efe5719-c999-4db6-a321-4691e2ca673e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:35.282540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1efe5719-c999-4db6-a321-4691e2ca673e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb3c9c8d94cb4a0aa7616e912fb54dd8', 'request_id': 'req-1efe5719-c999-4db6-a321-4691e2ca673e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:35.293101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1efe5719-c999-4db6-a321-4691e2ca673e admin admin] The token cache is being invalidate because project bb3c9c8d94cb4a0aa7616e912fb54dd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:35.293387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1efe5719-c999-4db6-a321-4691e2ca673e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:35.294038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-1efe5719-c999-4db6-a321-4691e2ca673e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:35.295578 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1354/2722] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:35 2022] DELETE /identity/v3/projects/bb3c9c8d94cb4a0aa7616e912fb54dd8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:28:35.301960 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d290437-3722-41e8-b65a-af32d2620e08 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:35.302493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d290437-3722-41e8-b65a-af32d2620e08 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:35.302493 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d290437-3722-41e8-b65a-af32d2620e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:35.321929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-0d290437-3722-41e8-b65a-af32d2620e08 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:35.323170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-0d290437-3722-41e8-b65a-af32d2620e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:35.383426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1369/2723] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:35.390926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b13e00-0ccc-4d1f-95ee-0bc355b46664 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:35.391280 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b13e00-0ccc-4d1f-95ee-0bc355b46664 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:35.391537 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b13e00-0ccc-4d1f-95ee-0bc355b46664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:35.414038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f5b13e00-0ccc-4d1f-95ee-0bc355b46664 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:35.415256 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f5b13e00-0ccc-4d1f-95ee-0bc355b46664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:35.424218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1355/2724] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:35.428977 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d290437-3722-41e8-b65a-af32d2620e08 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:35.438953 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d290437-3722-41e8-b65a-af32d2620e08 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:35.443570 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585092e-90b8-49c4-9063-fabd4a9613ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:35.444274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585092e-90b8-49c4-9063-fabd4a9613ca admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:35.444561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585092e-90b8-49c4-9063-fabd4a9613ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:35.444839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585092e-90b8-49c4-9063-fabd4a9613ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:35.445573 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585092e-90b8-49c4-9063-fabd4a9613ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:28:35.445881 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585092e-90b8-49c4-9063-fabd4a9613ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:35.447492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585092e-90b8-49c4-9063-fabd4a9613ca admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:35.460469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1370/2725] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:28:35.464846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b13e00-0ccc-4d1f-95ee-0bc355b46664 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:35.475094 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b13e00-0ccc-4d1f-95ee-0bc355b46664 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:35.479810 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d1101-e852-451d-8325-602b4cfbb8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:35.480501 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d1101-e852-451d-8325-602b4cfbb8d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:35.480786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d1101-e852-451d-8325-602b4cfbb8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:35.481062 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d1101-e852-451d-8325-602b4cfbb8d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:35.481763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d1101-e852-451d-8325-602b4cfbb8d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:35.483658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d1101-e852-451d-8325-602b4cfbb8d4 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:35.509454 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1356/2726] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:28:35 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:35.513672 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585092e-90b8-49c4-9063-fabd4a9613ca admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:35.524606 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585092e-90b8-49c4-9063-fabd4a9613ca admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:35.584583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685211b-8522-4e97-a96e-524ab779a0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:35.585652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685211b-8522-4e97-a96e-524ab779a0b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:35.585652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685211b-8522-4e97-a96e-524ab779a0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:35.586178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685211b-8522-4e97-a96e-524ab779a0b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:35.586592 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685211b-8522-4e97-a96e-524ab779a0b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:35.588773 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685211b-8522-4e97-a96e-524ab779a0b2 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:35.610131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1371/2727] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:28:35 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:35.613776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d1101-e852-451d-8325-602b4cfbb8d4 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:35.624292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d1101-e852-451d-8325-602b4cfbb8d4 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:35.629292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ae1d4-ca55-45ed-875c-efdb3312066d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:35.629954 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ae1d4-ca55-45ed-875c-efdb3312066d admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:35.630241 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ae1d4-ca55-45ed-875c-efdb3312066d admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:35.630842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ae1d4-ca55-45ed-875c-efdb3312066d admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:35.631570 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ae1d4-ca55-45ed-875c-efdb3312066d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:35.633197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ae1d4-ca55-45ed-875c-efdb3312066d admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:35.644534 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1357/2728] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:35.648782 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685211b-8522-4e97-a96e-524ab779a0b2 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:35.661117 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685211b-8522-4e97-a96e-524ab779a0b2 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:35.665912 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a36919-67a8-4eb0-8026-a6bb332af102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:35.666589 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a36919-67a8-4eb0-8026-a6bb332af102 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:35.666880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a36919-67a8-4eb0-8026-a6bb332af102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:35.667159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a36919-67a8-4eb0-8026-a6bb332af102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:35.667885 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a36919-67a8-4eb0-8026-a6bb332af102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:35.669544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a36919-67a8-4eb0-8026-a6bb332af102 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:35.680903 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1372/2729] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:35.685451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ae1d4-ca55-45ed-875c-efdb3312066d admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:35.695928 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ae1d4-ca55-45ed-875c-efdb3312066d admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:35.703187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abe20f-30ac-433a-9ab0-d727efc562a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:35.704631 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abe20f-30ac-433a-9ab0-d727efc562a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:35.704915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abe20f-30ac-433a-9ab0-d727efc562a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:35.705192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abe20f-30ac-433a-9ab0-d727efc562a8 admin admin] PATH_INFO: `/v3/projects/108ea45b5d974328b0ee94392d73cd2a/users/d1227219be5141c08c38c1e71a14de1e/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:35.720841 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40abe20f-30ac-433a-9ab0-d727efc562a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108ea45b5d974328b0ee94392d73cd2a, user_id=d1227219be5141c08c38c1e71a14de1e, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:35.723542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40abe20f-30ac-433a-9ab0-d727efc562a8 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:35.739069 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1358/2730] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:35 2022] PUT /identity/v3/projects/108ea45b5d974328b0ee94392d73cd2a/users/d1227219be5141c08c38c1e71a14de1e/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:28:35.752381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6133d-acfb-442e-bdcb-3c07233081f0 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:35.752704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6133d-acfb-442e-bdcb-3c07233081f0 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:35.752987 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6133d-acfb-442e-bdcb-3c07233081f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:35.780495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-14b6133d-acfb-442e-bdcb-3c07233081f0 None None] MFA Rules not processed for user `d1227219be5141c08c38c1e71a14de1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:35.781756 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-14b6133d-acfb-442e-bdcb-3c07233081f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:35.842044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:35.853546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1373/2731] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:35.857743 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abe20f-30ac-433a-9ab0-d727efc562a8 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:35.920764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abe20f-30ac-433a-9ab0-d727efc562a8 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:35.925589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e5032-bb5f-4ff4-8daf-7f323e7db677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:35.926587 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e5032-bb5f-4ff4-8daf-7f323e7db677 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:35.926871 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e5032-bb5f-4ff4-8daf-7f323e7db677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:35.927140 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e5032-bb5f-4ff4-8daf-7f323e7db677 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:35.927838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e5032-bb5f-4ff4-8daf-7f323e7db677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:35.930313 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e5032-bb5f-4ff4-8daf-7f323e7db677 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:35.943419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:35.961271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1359/2732] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:28:35 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:28:35.965348 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6133d-acfb-442e-bdcb-3c07233081f0 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:35.981191 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6133d-acfb-442e-bdcb-3c07233081f0 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:36.037860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f70471-3366-4742-a5b3-1bfb464a7aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:36.038535 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f70471-3366-4742-a5b3-1bfb464a7aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:36.038817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f70471-3366-4742-a5b3-1bfb464a7aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:36.039096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f70471-3366-4742-a5b3-1bfb464a7aeb admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:36.039779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f70471-3366-4742-a5b3-1bfb464a7aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:36.041759 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f70471-3366-4742-a5b3-1bfb464a7aeb admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:36.062690 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1374/2733] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:28:35 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:36.067296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e5032-bb5f-4ff4-8daf-7f323e7db677 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:36.078059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e5032-bb5f-4ff4-8daf-7f323e7db677 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:36.084039 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599b2fe-82f6-44bf-b984-15e6ae4ab0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:36.084876 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599b2fe-82f6-44bf-b984-15e6ae4ab0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:36.085164 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599b2fe-82f6-44bf-b984-15e6ae4ab0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:36.085443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599b2fe-82f6-44bf-b984-15e6ae4ab0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:36.086178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599b2fe-82f6-44bf-b984-15e6ae4ab0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:36.087877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599b2fe-82f6-44bf-b984-15e6ae4ab0b7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:36.099009 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1360/2734] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:36.103322 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f70471-3366-4742-a5b3-1bfb464a7aeb admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:36.113690 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f70471-3366-4742-a5b3-1bfb464a7aeb admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:36.119040 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ce067-e220-4e7e-a7ce-ccefe2cce1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:36.119938 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ce067-e220-4e7e-a7ce-ccefe2cce1fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:36.120248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ce067-e220-4e7e-a7ce-ccefe2cce1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:36.120550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ce067-e220-4e7e-a7ce-ccefe2cce1fc admin admin] PATH_INFO: `/v3/projects/79cd8f2354ec4e5aa52b3b3b4d4c6d1f/users/9abf73e8e25c4113abd2c5decbf924b1/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:36.136328 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ce067-e220-4e7e-a7ce-ccefe2cce1fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=79cd8f2354ec4e5aa52b3b3b4d4c6d1f, user_id=9abf73e8e25c4113abd2c5decbf924b1, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:36.138992 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ce067-e220-4e7e-a7ce-ccefe2cce1fc admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:36.153138 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1375/2735] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:36 2022] PUT /identity/v3/projects/79cd8f2354ec4e5aa52b3b3b4d4c6d1f/users/9abf73e8e25c4113abd2c5decbf924b1/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:36.157138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599b2fe-82f6-44bf-b984-15e6ae4ab0b7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:36.167621 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599b2fe-82f6-44bf-b984-15e6ae4ab0b7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:36.225702 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d582da6-14b7-4b5d-affb-8be6f306dd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:36.226378 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d582da6-14b7-4b5d-affb-8be6f306dd7c admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:36.226665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d582da6-14b7-4b5d-affb-8be6f306dd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:36.226944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d582da6-14b7-4b5d-affb-8be6f306dd7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:36.227681 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d582da6-14b7-4b5d-affb-8be6f306dd7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:36.229324 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d582da6-14b7-4b5d-affb-8be6f306dd7c admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:36.240004 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1361/2736] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:36.244242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ce067-e220-4e7e-a7ce-ccefe2cce1fc admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:36.255603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ce067-e220-4e7e-a7ce-ccefe2cce1fc admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:36.260496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0be07-b179-425c-b4d3-0190f7d8b4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:36.261399 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0be07-b179-425c-b4d3-0190f7d8b4f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:36.261689 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0be07-b179-425c-b4d3-0190f7d8b4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:36.261974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0be07-b179-425c-b4d3-0190f7d8b4f0 admin admin] PATH_INFO: `/v3/projects/79cd8f2354ec4e5aa52b3b3b4d4c6d1f/users/9abf73e8e25c4113abd2c5decbf924b1/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:36.263983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0be07-b179-425c-b4d3-0190f7d8b4f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79cd8f2354ec4e5aa52b3b3b4d4c6d1f, user_id=9abf73e8e25c4113abd2c5decbf924b1, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:36.266701 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0be07-b179-425c-b4d3-0190f7d8b4f0 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:36.277864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1376/2737] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:36 2022] PUT /identity/v3/projects/79cd8f2354ec4e5aa52b3b3b4d4c6d1f/users/9abf73e8e25c4113abd2c5decbf924b1/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:36.289045 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327ef5d-e682-487f-abbf-e1900c5c720c None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:36.289350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327ef5d-e682-487f-abbf-e1900c5c720c None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:36.289638 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327ef5d-e682-487f-abbf-e1900c5c720c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:36.320081 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-2327ef5d-e682-487f-abbf-e1900c5c720c None None] MFA Rules not processed for user `9abf73e8e25c4113abd2c5decbf924b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:36.321344 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-2327ef5d-e682-487f-abbf-e1900c5c720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:36.433143 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1362/2738] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:36.449192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0be07-b179-425c-b4d3-0190f7d8b4f0 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:36.451147 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-50f0be07-b179-425c-b4d3-0190f7d8b4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:36.525445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0be07-b179-425c-b4d3-0190f7d8b4f0 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:36.529461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc5b31-892f-4d7a-ac30-999b8e284d56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:36.530144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc5b31-892f-4d7a-ac30-999b8e284d56 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:36.530462 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc5b31-892f-4d7a-ac30-999b8e284d56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:36.530766 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc5b31-892f-4d7a-ac30-999b8e284d56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:36.542180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc5b31-892f-4d7a-ac30-999b8e284d56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:36.543735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc5b31-892f-4d7a-ac30-999b8e284d56 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:36.555192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1377/2739] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:36.951491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327ef5d-e682-487f-abbf-e1900c5c720c None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:37.021971 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327ef5d-e682-487f-abbf-e1900c5c720c None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:37.026735 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:37.027522 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:37.027807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:37.028170 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] PATH_INFO: `/v3/users/d1227219be5141c08c38c1e71a14de1e` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:37.029387 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1227219be5141c08c38c1e71a14de1e)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:37.031374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:37.034192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:37.089332 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1227219be5141c08c38c1e71a14de1e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:37.090026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1227219be5141c08c38c1e71a14de1e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:37.102273 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1227219be5141c08c38c1e71a14de1e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:37.123516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1227219be5141c08c38c1e71a14de1e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:37.124299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1227219be5141c08c38c1e71a14de1e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:37.143763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1363/2740] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:36 2022] DELETE /identity/v3/users/d1227219be5141c08c38c1e71a14de1e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:37.148475 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40187568-7001-44c3-914b-fa1cc82ad58d None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:37.148839 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40187568-7001-44c3-914b-fa1cc82ad58d None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:37.149180 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40187568-7001-44c3-914b-fa1cc82ad58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:37.168894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-40187568-7001-44c3-914b-fa1cc82ad58d None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:37.170216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-40187568-7001-44c3-914b-fa1cc82ad58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:37.217519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:37.234698 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1378/2741] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:37.245262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:37.246846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:37.317906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df765ef-3894-4866-a758-5ad1d9189c43 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:37.321944 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a89e5-1a17-474a-b352-2ff4e79fc812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:37.322720 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a89e5-1a17-474a-b352-2ff4e79fc812 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:37.323063 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a89e5-1a17-474a-b352-2ff4e79fc812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:37.323338 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a89e5-1a17-474a-b352-2ff4e79fc812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:37.333721 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a89e5-1a17-474a-b352-2ff4e79fc812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:37.335174 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a89e5-1a17-474a-b352-2ff4e79fc812 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:37.346098 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1364/2742] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:37.660360 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40187568-7001-44c3-914b-fa1cc82ad58d None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:37.663865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-40187568-7001-44c3-914b-fa1cc82ad58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:37.688842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40187568-7001-44c3-914b-fa1cc82ad58d None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:37.693704 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:37.695011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:37.695011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:37.695480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] PATH_INFO: `/v3/projects/108ea45b5d974328b0ee94392d73cd2a` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:37.696648 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] RBAC: Authorizing `identity:delete_project(project_id=108ea45b5d974328b0ee94392d73cd2a)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:37.699083 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:37.770186 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108ea45b5d974328b0ee94392d73cd2a', 'request_id': 'req-8076159d-cd8e-4c50-833a-473945a3a90b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:37.777623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108ea45b5d974328b0ee94392d73cd2a', 'request_id': 'req-8076159d-cd8e-4c50-833a-473945a3a90b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:37.785997 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108ea45b5d974328b0ee94392d73cd2a', 'request_id': 'req-8076159d-cd8e-4c50-833a-473945a3a90b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:37.800283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] The token cache is being invalidate because project 108ea45b5d974328b0ee94392d73cd2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:37.800722 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:37.801626 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:37.803587 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1379/2743] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:37 2022] DELETE /identity/v3/projects/108ea45b5d974328b0ee94392d73cd2a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:37.807604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a89e5-1a17-474a-b352-2ff4e79fc812 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:37.819170 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a89e5-1a17-474a-b352-2ff4e79fc812 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:37.919498 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db800b2a-fce7-4f28-a838-b753d0293705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:37.920830 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db800b2a-fce7-4f28-a838-b753d0293705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:37.921259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db800b2a-fce7-4f28-a838-b753d0293705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:37.921637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db800b2a-fce7-4f28-a838-b753d0293705 admin admin] PATH_INFO: `/v3/users/9abf73e8e25c4113abd2c5decbf924b1` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:37.923635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db800b2a-fce7-4f28-a838-b753d0293705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9abf73e8e25c4113abd2c5decbf924b1)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:37.926112 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db800b2a-fce7-4f28-a838-b753d0293705 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:37.967667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:38.003165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-db800b2a-fce7-4f28-a838-b753d0293705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9abf73e8e25c4113abd2c5decbf924b1'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:38.003948 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-db800b2a-fce7-4f28-a838-b753d0293705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9abf73e8e25c4113abd2c5decbf924b1'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:38.025860 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-db800b2a-fce7-4f28-a838-b753d0293705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9abf73e8e25c4113abd2c5decbf924b1'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:38.046214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-db800b2a-fce7-4f28-a838-b753d0293705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9abf73e8e25c4113abd2c5decbf924b1'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:38.047356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-db800b2a-fce7-4f28-a838-b753d0293705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9abf73e8e25c4113abd2c5decbf924b1'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:38.066351 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1365/2744] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:37 2022] DELETE /identity/v3/users/9abf73e8e25c4113abd2c5decbf924b1 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:28:38.409260 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:38.411513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:38.426193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076159d-cd8e-4c50-833a-473945a3a90b admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:38.488273 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54227fb-5a4a-48f6-a795-d2936783caf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:38.489151 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54227fb-5a4a-48f6-a795-d2936783caf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:38.489437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54227fb-5a4a-48f6-a795-d2936783caf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:38.489717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54227fb-5a4a-48f6-a795-d2936783caf9 admin admin] PATH_INFO: `/v3/projects/79cd8f2354ec4e5aa52b3b3b4d4c6d1f` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:38.497669 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54227fb-5a4a-48f6-a795-d2936783caf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79cd8f2354ec4e5aa52b3b3b4d4c6d1f)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:38.499630 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54227fb-5a4a-48f6-a795-d2936783caf9 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:38.551725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d54227fb-5a4a-48f6-a795-d2936783caf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79cd8f2354ec4e5aa52b3b3b4d4c6d1f', 'request_id': 'req-d54227fb-5a4a-48f6-a795-d2936783caf9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:38.557364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d54227fb-5a4a-48f6-a795-d2936783caf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79cd8f2354ec4e5aa52b3b3b4d4c6d1f', 'request_id': 'req-d54227fb-5a4a-48f6-a795-d2936783caf9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:38.563736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d54227fb-5a4a-48f6-a795-d2936783caf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79cd8f2354ec4e5aa52b3b3b4d4c6d1f', 'request_id': 'req-d54227fb-5a4a-48f6-a795-d2936783caf9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:38.576762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d54227fb-5a4a-48f6-a795-d2936783caf9 admin admin] The token cache is being invalidate because project 79cd8f2354ec4e5aa52b3b3b4d4c6d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:38.577061 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d54227fb-5a4a-48f6-a795-d2936783caf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:38.577728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-d54227fb-5a4a-48f6-a795-d2936783caf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:38.579845 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1380/2745] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:38 2022] DELETE /identity/v3/projects/79cd8f2354ec4e5aa52b3b3b4d4c6d1f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:38.584780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d78e2f-d992-4d3b-8bea-d44741d231ac None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:38.585283 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d78e2f-d992-4d3b-8bea-d44741d231ac None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:38.585382 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d78e2f-d992-4d3b-8bea-d44741d231ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:38.607432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-66d78e2f-d992-4d3b-8bea-d44741d231ac None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:38.610016 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-66d78e2f-d992-4d3b-8bea-d44741d231ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:38.673622 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1366/2746] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:38.681637 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35b990-6df9-47d4-aa8a-62da728497a5 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:38.682384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35b990-6df9-47d4-aa8a-62da728497a5 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:38.682384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35b990-6df9-47d4-aa8a-62da728497a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:38.689850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:38.702605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-1e35b990-6df9-47d4-aa8a-62da728497a5 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:38.704655 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1e35b990-6df9-47d4-aa8a-62da728497a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:38.713001 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1381/2747] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:28:38.717295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d78e2f-d992-4d3b-8bea-d44741d231ac None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:38.729937 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d78e2f-d992-4d3b-8bea-d44741d231ac None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:38.734628 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cd35c-128f-4f2a-b4be-fc73b11c6ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:38.735305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cd35c-128f-4f2a-b4be-fc73b11c6ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:38.735595 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cd35c-128f-4f2a-b4be-fc73b11c6ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:38.735873 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cd35c-128f-4f2a-b4be-fc73b11c6ea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:38.736663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cd35c-128f-4f2a-b4be-fc73b11c6ea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:28:38.736979 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cd35c-128f-4f2a-b4be-fc73b11c6ea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:38.738640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cd35c-128f-4f2a-b4be-fc73b11c6ea9 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:38.751206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1367/2748] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:28:38.755740 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35b990-6df9-47d4-aa8a-62da728497a5 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:38.765501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35b990-6df9-47d4-aa8a-62da728497a5 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:38.770753 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bd417-d094-4adc-b98a-5a82b754b4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:38.771428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bd417-d094-4adc-b98a-5a82b754b4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:38.771712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bd417-d094-4adc-b98a-5a82b754b4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:38.772007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bd417-d094-4adc-b98a-5a82b754b4c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:38.772734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bd417-d094-4adc-b98a-5a82b754b4c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:38.774638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bd417-d094-4adc-b98a-5a82b754b4c4 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:38.802554 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1382/2749] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:28:38 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:38.807316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cd35c-128f-4f2a-b4be-fc73b11c6ea9 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:38.817892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cd35c-128f-4f2a-b4be-fc73b11c6ea9 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:38.886604 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ab5d4-cade-4ae7-bece-1bc9381f5cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:38.887007 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ab5d4-cade-4ae7-bece-1bc9381f5cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:38.887295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ab5d4-cade-4ae7-bece-1bc9381f5cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:38.887574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ab5d4-cade-4ae7-bece-1bc9381f5cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:38.888353 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ab5d4-cade-4ae7-bece-1bc9381f5cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:38.890284 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ab5d4-cade-4ae7-bece-1bc9381f5cb7 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:38.911415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1368/2750] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:28:38 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:38.915695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bd417-d094-4adc-b98a-5a82b754b4c4 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:38.926500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bd417-d094-4adc-b98a-5a82b754b4c4 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:38.931378 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86476d-6fe9-4bdf-844d-c078639d5c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:38.932051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86476d-6fe9-4bdf-844d-c078639d5c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:38.932362 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86476d-6fe9-4bdf-844d-c078639d5c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:38.932639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86476d-6fe9-4bdf-844d-c078639d5c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:38.933364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86476d-6fe9-4bdf-844d-c078639d5c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:38.934980 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86476d-6fe9-4bdf-844d-c078639d5c87 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:38.947361 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1383/2751] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:38.951494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ab5d4-cade-4ae7-bece-1bc9381f5cb7 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:38.962745 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ab5d4-cade-4ae7-bece-1bc9381f5cb7 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:38.968844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a107cee-704e-4a9a-a0cf-99bc0583e117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:38.970047 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a107cee-704e-4a9a-a0cf-99bc0583e117 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:38.970547 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a107cee-704e-4a9a-a0cf-99bc0583e117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:38.971024 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a107cee-704e-4a9a-a0cf-99bc0583e117 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:38.972381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a107cee-704e-4a9a-a0cf-99bc0583e117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:38.975287 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a107cee-704e-4a9a-a0cf-99bc0583e117 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:38.986649 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1369/2752] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:38.991415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86476d-6fe9-4bdf-844d-c078639d5c87 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:39.003045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86476d-6fe9-4bdf-844d-c078639d5c87 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:39.008427 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec50f94-5868-4fd1-b7dc-7d15bb6d378f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:39.009579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec50f94-5868-4fd1-b7dc-7d15bb6d378f admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:39.009986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec50f94-5868-4fd1-b7dc-7d15bb6d378f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:39.010285 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec50f94-5868-4fd1-b7dc-7d15bb6d378f admin admin] PATH_INFO: `/v3/projects/1f4d52c8c4b14264bacb64bb1e61ae69/users/abf956afc66e42e6bb7ea120fe341743/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:39.027742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec50f94-5868-4fd1-b7dc-7d15bb6d378f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4d52c8c4b14264bacb64bb1e61ae69, user_id=abf956afc66e42e6bb7ea120fe341743, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:39.030556 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec50f94-5868-4fd1-b7dc-7d15bb6d378f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:39.053509 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1384/2753] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:38 2022] PUT /identity/v3/projects/1f4d52c8c4b14264bacb64bb1e61ae69/users/abf956afc66e42e6bb7ea120fe341743/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:39.060647 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dfa81-ec2c-4a3c-8af3-efe93e586c91 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:39.061151 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dfa81-ec2c-4a3c-8af3-efe93e586c91 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:39.061696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dfa81-ec2c-4a3c-8af3-efe93e586c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:39.088902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-383dfa81-ec2c-4a3c-8af3-efe93e586c91 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:39.091335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-383dfa81-ec2c-4a3c-8af3-efe93e586c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:39.098443 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:39.176696 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1370/2754] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:39.187208 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec50f94-5868-4fd1-b7dc-7d15bb6d378f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:39.189676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cec50f94-5868-4fd1-b7dc-7d15bb6d378f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:39.262607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec50f94-5868-4fd1-b7dc-7d15bb6d378f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:39.267979 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da9b24-7d35-450d-b6ee-5dccdde29417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:39.268677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da9b24-7d35-450d-b6ee-5dccdde29417 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:39.268961 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da9b24-7d35-450d-b6ee-5dccdde29417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:39.269304 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da9b24-7d35-450d-b6ee-5dccdde29417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:39.280546 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da9b24-7d35-450d-b6ee-5dccdde29417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:39.283401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da9b24-7d35-450d-b6ee-5dccdde29417 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:39.294060 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1385/2755] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:28:40.074137 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e2220-8b55-4814-a8f7-c9a1a20feb9e None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:40.074691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e2220-8b55-4814-a8f7-c9a1a20feb9e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:40.074691 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e2220-8b55-4814-a8f7-c9a1a20feb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:40.103603 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-4f2e2220-8b55-4814-a8f7-c9a1a20feb9e None None] MFA Rules not processed for user `abf956afc66e42e6bb7ea120fe341743`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:40.104914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-4f2e2220-8b55-4814-a8f7-c9a1a20feb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:40.178453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1371/2756] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:40.182875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da9b24-7d35-450d-b6ee-5dccdde29417 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:40.199024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da9b24-7d35-450d-b6ee-5dccdde29417 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:40.207835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d0d95-553b-4c58-a935-b8cc6befe4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:40.209145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d0d95-553b-4c58-a935-b8cc6befe4ba admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:40.209751 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d0d95-553b-4c58-a935-b8cc6befe4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:40.210332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d0d95-553b-4c58-a935-b8cc6befe4ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:40.211734 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d0d95-553b-4c58-a935-b8cc6befe4ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:40.215480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d0d95-553b-4c58-a935-b8cc6befe4ba admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:40.243178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1386/2757] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:28:40 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:40.247546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e2220-8b55-4814-a8f7-c9a1a20feb9e None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:40.259105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e2220-8b55-4814-a8f7-c9a1a20feb9e None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:40.318223 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d805b2e-789c-4c3c-9966-0c7f7dd75890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:40.318906 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d805b2e-789c-4c3c-9966-0c7f7dd75890 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:40.319196 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d805b2e-789c-4c3c-9966-0c7f7dd75890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:40.319476 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d805b2e-789c-4c3c-9966-0c7f7dd75890 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:40.320230 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d805b2e-789c-4c3c-9966-0c7f7dd75890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:40.322237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d805b2e-789c-4c3c-9966-0c7f7dd75890 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:40.347023 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1372/2758] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:28:40 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:40.351127 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d0d95-553b-4c58-a935-b8cc6befe4ba admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:40.361778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d0d95-553b-4c58-a935-b8cc6befe4ba admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:40.366526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2067fd-c118-4caa-8a7d-8b0965217176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:40.367201 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2067fd-c118-4caa-8a7d-8b0965217176 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:40.367490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2067fd-c118-4caa-8a7d-8b0965217176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:40.367772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2067fd-c118-4caa-8a7d-8b0965217176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:40.368557 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2067fd-c118-4caa-8a7d-8b0965217176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:40.370166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2067fd-c118-4caa-8a7d-8b0965217176 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:40.381712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1387/2759] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:40.386115 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d805b2e-789c-4c3c-9966-0c7f7dd75890 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:40.397932 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d805b2e-789c-4c3c-9966-0c7f7dd75890 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:40.402629 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a3beb-8947-46ef-8e6a-325df89328de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:40.403526 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a3beb-8947-46ef-8e6a-325df89328de admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:40.403813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a3beb-8947-46ef-8e6a-325df89328de admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:40.404121 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a3beb-8947-46ef-8e6a-325df89328de admin admin] PATH_INFO: `/v3/projects/c66f4217188b4b579f055612f36ef036/users/c3498d5c874c40a7b138b1c790bd84fc/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:40.419379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a3beb-8947-46ef-8e6a-325df89328de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c66f4217188b4b579f055612f36ef036, user_id=c3498d5c874c40a7b138b1c790bd84fc, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:40.422080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a3beb-8947-46ef-8e6a-325df89328de admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:40.435782 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1373/2760] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:40 2022] PUT /identity/v3/projects/c66f4217188b4b579f055612f36ef036/users/c3498d5c874c40a7b138b1c790bd84fc/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:40.440403 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2067fd-c118-4caa-8a7d-8b0965217176 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:40.450683 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2067fd-c118-4caa-8a7d-8b0965217176 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:40.509711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038560c3-603c-4171-89de-f186c0c8f203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:40.510397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038560c3-603c-4171-89de-f186c0c8f203 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:40.510687 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038560c3-603c-4171-89de-f186c0c8f203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:40.510969 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038560c3-603c-4171-89de-f186c0c8f203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:40.511705 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038560c3-603c-4171-89de-f186c0c8f203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:40.513356 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038560c3-603c-4171-89de-f186c0c8f203 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:40.527269 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1388/2761] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:40.528510 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a3beb-8947-46ef-8e6a-325df89328de admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:40.535753 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:40.539573 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a3beb-8947-46ef-8e6a-325df89328de admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:40.545149 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2868e-1ae4-437e-9f14-e385a0b33f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:40.546050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2868e-1ae4-437e-9f14-e385a0b33f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:40.546337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2868e-1ae4-437e-9f14-e385a0b33f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:40.546613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2868e-1ae4-437e-9f14-e385a0b33f80 admin admin] PATH_INFO: `/v3/projects/c66f4217188b4b579f055612f36ef036/users/c3498d5c874c40a7b138b1c790bd84fc/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:40.548710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2868e-1ae4-437e-9f14-e385a0b33f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c66f4217188b4b579f055612f36ef036, user_id=c3498d5c874c40a7b138b1c790bd84fc, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:40.551698 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2868e-1ae4-437e-9f14-e385a0b33f80 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:40.562440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1374/2762] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:40 2022] PUT /identity/v3/projects/c66f4217188b4b579f055612f36ef036/users/c3498d5c874c40a7b138b1c790bd84fc/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:41.213827 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ee154-97da-4a8d-b3fc-d8ab320c75d7 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:41.214424 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ee154-97da-4a8d-b3fc-d8ab320c75d7 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:41.214526 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ee154-97da-4a8d-b3fc-d8ab320c75d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:41.228618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:41.245983 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-e96ee154-97da-4a8d-b3fc-d8ab320c75d7 None None] MFA Rules not processed for user `c3498d5c874c40a7b138b1c790bd84fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:41.247495 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e96ee154-97da-4a8d-b3fc-d8ab320c75d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:41.327398 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1389/2763] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:41.339195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2868e-1ae4-437e-9f14-e385a0b33f80 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:41.340960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ccc2868e-1ae4-437e-9f14-e385a0b33f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:41.408413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2868e-1ae4-437e-9f14-e385a0b33f80 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:41.412235 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbafbac-1774-4e94-b59b-255fb5bc6639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:41.412896 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbafbac-1774-4e94-b59b-255fb5bc6639 service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:41.413185 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbafbac-1774-4e94-b59b-255fb5bc6639 service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:41.413468 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbafbac-1774-4e94-b59b-255fb5bc6639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:41.470995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbafbac-1774-4e94-b59b-255fb5bc6639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:41.472490 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbafbac-1774-4e94-b59b-255fb5bc6639 service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:41.481517 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1375/2764] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:42.835501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ee154-97da-4a8d-b3fc-d8ab320c75d7 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:42.845711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ee154-97da-4a8d-b3fc-d8ab320c75d7 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:42.849875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bead916-c193-479e-b4af-9cb71f68a3eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:42.850548 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bead916-c193-479e-b4af-9cb71f68a3eb service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:42.850832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bead916-c193-479e-b4af-9cb71f68a3eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:42.851106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bead916-c193-479e-b4af-9cb71f68a3eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:42.861552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bead916-c193-479e-b4af-9cb71f68a3eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:42.863039 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bead916-c193-479e-b4af-9cb71f68a3eb service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:42.874532 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1390/2765] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:28:51.477440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=51,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:52.869212 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=37 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:53.090049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0aee5-7ecb-46ce-a38a-131f210ac442 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:53.090709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0aee5-7ecb-46ce-a38a-131f210ac442 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:53.090709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0aee5-7ecb-46ce-a38a-131f210ac442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:53.109937 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-a5b0aee5-7ecb-46ce-a38a-131f210ac442 None None] MFA Rules not processed for user `de859eecbef7498289f132a047e6e5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:53.111764 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-a5b0aee5-7ecb-46ce-a38a-131f210ac442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:53.176254 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1376/2766] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:56.734644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f7e7-4614-4e7d-ae1d-e66831fa4ce4 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:56.734644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f7e7-4614-4e7d-ae1d-e66831fa4ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:56.734644 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98f7e7-4614-4e7d-ae1d-e66831fa4ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:56.756058 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-3a98f7e7-4614-4e7d-ae1d-e66831fa4ce4 None None] MFA Rules not processed for user `6a113fe9b988458d8e8898c5c9a8d02d`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:56.757472 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3a98f7e7-4614-4e7d-ae1d-e66831fa4ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:56.826921 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0aee5-7ecb-46ce-a38a-131f210ac442 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:56.852270 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1391/2767] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:56.906267 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0aee5-7ecb-46ce-a38a-131f210ac442 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:56.912099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:56.913104 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:56.913453 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:56.913963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] PATH_INFO: `/v3/users/abf956afc66e42e6bb7ea120fe341743` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:56.915517 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf956afc66e42e6bb7ea120fe341743)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:56.917995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:56.977452 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf956afc66e42e6bb7ea120fe341743'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:56.978574 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf956afc66e42e6bb7ea120fe341743'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:56.987061 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf956afc66e42e6bb7ea120fe341743'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:57.001834 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf956afc66e42e6bb7ea120fe341743'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:57.002561 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf956afc66e42e6bb7ea120fe341743'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:57.017434 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1377/2768] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:56 2022] DELETE /identity/v3/users/abf956afc66e42e6bb7ea120fe341743 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:57.417575 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f7e7-4614-4e7d-ae1d-e66831fa4ce4 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:57.439349 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98f7e7-4614-4e7d-ae1d-e66831fa4ce4 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:57.576865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:57.577861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:57.578156 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:57.578438 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] PATH_INFO: `/v3/projects/1f4d52c8c4b14264bacb64bb1e61ae69` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:57.580423 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4d52c8c4b14264bacb64bb1e61ae69)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:57.582497 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:57.663880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4d52c8c4b14264bacb64bb1e61ae69', 'request_id': 'req-4e5f74dc-db83-43c2-8b81-3eb522409e3b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:57.674636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4d52c8c4b14264bacb64bb1e61ae69', 'request_id': 'req-4e5f74dc-db83-43c2-8b81-3eb522409e3b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:57.690331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4d52c8c4b14264bacb64bb1e61ae69', 'request_id': 'req-4e5f74dc-db83-43c2-8b81-3eb522409e3b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:57.711248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] The token cache is being invalidate because project 1f4d52c8c4b14264bacb64bb1e61ae69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:57.711831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:57.712717 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:57.715254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1392/2769] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:57 2022] DELETE /identity/v3/projects/1f4d52c8c4b14264bacb64bb1e61ae69 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:57.720108 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:57.722026 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:57.741806 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd34ba-7305-4916-812d-0ff230c01b63 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:57.789970 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:57.861722 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590dd05c-5799-4f4f-aafe-779784ee2202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:57.864777 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dd05c-5799-4f4f-aafe-779784ee2202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:57.866170 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dd05c-5799-4f4f-aafe-779784ee2202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:57.866763 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dd05c-5799-4f4f-aafe-779784ee2202 admin admin] PATH_INFO: `/v3/users/c3498d5c874c40a7b138b1c790bd84fc` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:57.868350 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590dd05c-5799-4f4f-aafe-779784ee2202 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3498d5c874c40a7b138b1c790bd84fc)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:57.870789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590dd05c-5799-4f4f-aafe-779784ee2202 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:57.966171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-590dd05c-5799-4f4f-aafe-779784ee2202 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3498d5c874c40a7b138b1c790bd84fc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:57.967138 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-590dd05c-5799-4f4f-aafe-779784ee2202 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3498d5c874c40a7b138b1c790bd84fc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:57.978220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-590dd05c-5799-4f4f-aafe-779784ee2202 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3498d5c874c40a7b138b1c790bd84fc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:57.991907 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-590dd05c-5799-4f4f-aafe-779784ee2202 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3498d5c874c40a7b138b1c790bd84fc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:57.992843 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-590dd05c-5799-4f4f-aafe-779784ee2202 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3498d5c874c40a7b138b1c790bd84fc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:58.008993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1378/2770] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:28:57 2022] DELETE /identity/v3/users/c3498d5c874c40a7b138b1c790bd84fc => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:58.252581 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:58.254043 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:58.266923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f74dc-db83-43c2-8b81-3eb522409e3b admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:58.300652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:58.327785 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8aac47-0074-4783-9602-b012fb7d94e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:58.328697 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8aac47-0074-4783-9602-b012fb7d94e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:58.329500 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8aac47-0074-4783-9602-b012fb7d94e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:58.329779 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8aac47-0074-4783-9602-b012fb7d94e9 admin admin] PATH_INFO: `/v3/projects/c66f4217188b4b579f055612f36ef036` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:58.330971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8aac47-0074-4783-9602-b012fb7d94e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c66f4217188b4b579f055612f36ef036)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:58.332948 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8aac47-0074-4783-9602-b012fb7d94e9 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:58.378583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-ab8aac47-0074-4783-9602-b012fb7d94e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c66f4217188b4b579f055612f36ef036', 'request_id': 'req-ab8aac47-0074-4783-9602-b012fb7d94e9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:58.384005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-ab8aac47-0074-4783-9602-b012fb7d94e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c66f4217188b4b579f055612f36ef036', 'request_id': 'req-ab8aac47-0074-4783-9602-b012fb7d94e9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:58.390311 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-ab8aac47-0074-4783-9602-b012fb7d94e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c66f4217188b4b579f055612f36ef036', 'request_id': 'req-ab8aac47-0074-4783-9602-b012fb7d94e9'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:58.404868 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-ab8aac47-0074-4783-9602-b012fb7d94e9 admin admin] The token cache is being invalidate because project c66f4217188b4b579f055612f36ef036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:28:58.405198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-ab8aac47-0074-4783-9602-b012fb7d94e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:58.405832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-ab8aac47-0074-4783-9602-b012fb7d94e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:28:58.407926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1393/2771] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:28:58 2022] DELETE /identity/v3/projects/c66f4217188b4b579f055612f36ef036 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:58.413221 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5a006-1942-4a3c-bf10-e6afa53f8fb7 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:58.413548 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5a006-1942-4a3c-bf10-e6afa53f8fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:58.413836 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5a006-1942-4a3c-bf10-e6afa53f8fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:58.432794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-57f5a006-1942-4a3c-bf10-e6afa53f8fb7 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:58.434014 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-57f5a006-1942-4a3c-bf10-e6afa53f8fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:58.494208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1379/2772] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:58.503544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e006c08-1355-4180-9b5b-607671bfc6fa None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:58.503761 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e006c08-1355-4180-9b5b-607671bfc6fa None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:58.504045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e006c08-1355-4180-9b5b-607671bfc6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:58.522237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-7e006c08-1355-4180-9b5b-607671bfc6fa None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:58.523486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7e006c08-1355-4180-9b5b-607671bfc6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:58.531515 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1394/2773] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:58.536455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5a006-1942-4a3c-bf10-e6afa53f8fb7 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:58.546295 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5a006-1942-4a3c-bf10-e6afa53f8fb7 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:58.551479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d971c-c33c-4482-9202-006aaf8a662e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:58.552427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d971c-c33c-4482-9202-006aaf8a662e admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:58.552712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d971c-c33c-4482-9202-006aaf8a662e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:58.552990 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d971c-c33c-4482-9202-006aaf8a662e admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:58.553724 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d971c-c33c-4482-9202-006aaf8a662e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:28:58.554035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d971c-c33c-4482-9202-006aaf8a662e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:58.555640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d971c-c33c-4482-9202-006aaf8a662e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:58.567966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1380/2774] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:28:58.572424 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e006c08-1355-4180-9b5b-607671bfc6fa None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:58.582425 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e006c08-1355-4180-9b5b-607671bfc6fa None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:58.587174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99a34e-98c5-48f2-914f-0e4c6903ac5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:58.587830 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99a34e-98c5-48f2-914f-0e4c6903ac5e admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:58.588158 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99a34e-98c5-48f2-914f-0e4c6903ac5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:58.588445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99a34e-98c5-48f2-914f-0e4c6903ac5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:58.589137 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99a34e-98c5-48f2-914f-0e4c6903ac5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:58.591055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99a34e-98c5-48f2-914f-0e4c6903ac5e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:58.621681 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1395/2775] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:28:58 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:58.625780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d971c-c33c-4482-9202-006aaf8a662e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:58.636266 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d971c-c33c-4482-9202-006aaf8a662e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:58.665680 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:28:58.695499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11673532-4ead-41cc-850a-f9b1ef443884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:58.696201 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11673532-4ead-41cc-850a-f9b1ef443884 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:58.696491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11673532-4ead-41cc-850a-f9b1ef443884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:58.696770 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11673532-4ead-41cc-850a-f9b1ef443884 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:58.697467 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11673532-4ead-41cc-850a-f9b1ef443884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:58.699389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11673532-4ead-41cc-850a-f9b1ef443884 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:58.719229 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1381/2776] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:28:58 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:28:58.723675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99a34e-98c5-48f2-914f-0e4c6903ac5e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:58.733958 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99a34e-98c5-48f2-914f-0e4c6903ac5e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:58.739614 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039857e-fc90-4481-be81-c8f464d39548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:58.739871 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039857e-fc90-4481-be81-c8f464d39548 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:58.740195 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039857e-fc90-4481-be81-c8f464d39548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:58.740476 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039857e-fc90-4481-be81-c8f464d39548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:58.741202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039857e-fc90-4481-be81-c8f464d39548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:58.742805 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039857e-fc90-4481-be81-c8f464d39548 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:58.754050 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1396/2777] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:58.758038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11673532-4ead-41cc-850a-f9b1ef443884 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:58.768773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11673532-4ead-41cc-850a-f9b1ef443884 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:58.773491 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dcf09-9c68-4125-858a-beece4e1c28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:58.774207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dcf09-9c68-4125-858a-beece4e1c28a admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:58.774494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dcf09-9c68-4125-858a-beece4e1c28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:58.774807 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dcf09-9c68-4125-858a-beece4e1c28a admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:58.775540 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75dcf09-9c68-4125-858a-beece4e1c28a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:58.777170 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75dcf09-9c68-4125-858a-beece4e1c28a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:58.788017 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1382/2778] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:28:58.792283 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039857e-fc90-4481-be81-c8f464d39548 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:58.803272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039857e-fc90-4481-be81-c8f464d39548 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:58.808034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d9b73-058c-41d4-9b2a-885c801b228c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:58.808945 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d9b73-058c-41d4-9b2a-885c801b228c admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:58.809233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d9b73-058c-41d4-9b2a-885c801b228c admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:58.809514 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d9b73-058c-41d4-9b2a-885c801b228c admin admin] PATH_INFO: `/v3/projects/d18781fbe5bb4dafa7f9b128fc248b83/users/a990654283404659b7b38ea3a570becb/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:58.825652 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d9b73-058c-41d4-9b2a-885c801b228c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18781fbe5bb4dafa7f9b128fc248b83, user_id=a990654283404659b7b38ea3a570becb, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:58.828389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d9b73-058c-41d4-9b2a-885c801b228c admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:58.846051 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1397/2779] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:28:58 2022] PUT /identity/v3/projects/d18781fbe5bb4dafa7f9b128fc248b83/users/a990654283404659b7b38ea3a570becb/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:28:58.856243 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdf99c-8de0-4bb6-a686-ce6ff3ef953f None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:58.856684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdf99c-8de0-4bb6-a686-ce6ff3ef953f None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:58.857071 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdf99c-8de0-4bb6-a686-ce6ff3ef953f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:58.890290 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c1cdf99c-8de0-4bb6-a686-ce6ff3ef953f None None] MFA Rules not processed for user `a990654283404659b7b38ea3a570becb`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:28:58.893355 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c1cdf99c-8de0-4bb6-a686-ce6ff3ef953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:58.973288 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1383/2780] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:28:58 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:28:58.984729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d9b73-058c-41d4-9b2a-885c801b228c admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:28:58.987216 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c98d9b73-058c-41d4-9b2a-885c801b228c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:28:59.060686 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d9b73-058c-41d4-9b2a-885c801b228c admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:28:59.066630 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c8b7a-b607-45df-a63b-b20f73bfd9cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:28:59.067549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8b7a-b607-45df-a63b-b20f73bfd9cb service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:28:59.068029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8b7a-b607-45df-a63b-b20f73bfd9cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:28:59.068499 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8b7a-b607-45df-a63b-b20f73bfd9cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:28:59.080560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c8b7a-b607-45df-a63b-b20f73bfd9cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:28:59.082583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c8b7a-b607-45df-a63b-b20f73bfd9cb service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:28:59.093006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1398/2781] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:28:58 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:29:08.971021 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:29:09.088792 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:29:33.977507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdf99c-8de0-4bb6-a686-ce6ff3ef953f None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:29:34.042133 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdf99c-8de0-4bb6-a686-ce6ff3ef953f None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:29:34.046678 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:29:34.047556 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:29:34.047847 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:29:34.048159 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] PATH_INFO: `/v3/users/a990654283404659b7b38ea3a570becb` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:29:34.049362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a990654283404659b7b38ea3a570becb)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:29:34.051320 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:29:34.100648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a990654283404659b7b38ea3a570becb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:29:34.101300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a990654283404659b7b38ea3a570becb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:29:34.108454 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a990654283404659b7b38ea3a570becb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:29:34.120545 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a990654283404659b7b38ea3a570becb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:29:34.121269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a990654283404659b7b38ea3a570becb'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:29:34.135100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1384/2782] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:29:33 2022] DELETE /identity/v3/users/a990654283404659b7b38ea3a570becb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:29:34.139662 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c390-691c-472e-9f12-4f92f7ba7cc6 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:29:34.140187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c390-691c-472e-9f12-4f92f7ba7cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:29:34.140293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c390-691c-472e-9f12-4f92f7ba7cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:29:34.160001 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-cdd6c390-691c-472e-9f12-4f92f7ba7cc6 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:29:34.161272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cdd6c390-691c-472e-9f12-4f92f7ba7cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:29:34.227912 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1399/2783] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:29:34.237693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:29:34.239099 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:29:34.300859 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0640a6d-6528-46c7-aeec-bbcd1768b01d admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:29:34.304884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480ca88-ae36-40ad-9b8e-9e7f1a11f8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:29:34.305541 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480ca88-ae36-40ad-9b8e-9e7f1a11f8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:29:34.305829 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480ca88-ae36-40ad-9b8e-9e7f1a11f8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:29:34.306112 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480ca88-ae36-40ad-9b8e-9e7f1a11f8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:29:34.316712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480ca88-ae36-40ad-9b8e-9e7f1a11f8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:29:34.318376 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480ca88-ae36-40ad-9b8e-9e7f1a11f8d8 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:29:34.328368 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1385/2784] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:29:34.603636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6c390-691c-472e-9f12-4f92f7ba7cc6 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:29:34.605125 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cdd6c390-691c-472e-9f12-4f92f7ba7cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:29:34.618583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6c390-691c-472e-9f12-4f92f7ba7cc6 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:29:34.624492 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:29:34.626513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:29:34.626513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:29:34.626513 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] PATH_INFO: `/v3/projects/d18781fbe5bb4dafa7f9b128fc248b83` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:29:34.628994 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d18781fbe5bb4dafa7f9b128fc248b83)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:29:34.632787 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:29:34.698835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd18781fbe5bb4dafa7f9b128fc248b83', 'request_id': 'req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:29:34.706207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd18781fbe5bb4dafa7f9b128fc248b83', 'request_id': 'req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:29:34.713318 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd18781fbe5bb4dafa7f9b128fc248b83', 'request_id': 'req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:29:34.727347 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] The token cache is being invalidate because project d18781fbe5bb4dafa7f9b128fc248b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:29:34.727645 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:29:34.728374 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:29:34.730287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1400/2785] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:29:34 2022] DELETE /identity/v3/projects/d18781fbe5bb4dafa7f9b128fc248b83 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:29:44.324289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:29:44.724437 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:29:45.670421 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480ca88-ae36-40ad-9b8e-9e7f1a11f8d8 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:29:45.671897 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e480ca88-ae36-40ad-9b8e-9e7f1a11f8d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:29:45.732299 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480ca88-ae36-40ad-9b8e-9e7f1a11f8d8 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:29:45.781233 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33d444-e011-46c6-8a2d-c14e9f770dcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:29:45.781910 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33d444-e011-46c6-8a2d-c14e9f770dcd service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:29:45.782202 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33d444-e011-46c6-8a2d-c14e9f770dcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:29:45.782483 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33d444-e011-46c6-8a2d-c14e9f770dcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:29:45.784399 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-cb33d444-e011-46c6-8a2d-c14e9f770dcd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:29:45.855440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33d444-e011-46c6-8a2d-c14e9f770dcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:29:45.856858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33d444-e011-46c6-8a2d-c14e9f770dcd service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:29:45.869647 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1386/2786] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:29:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:29:52.726946 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:29:52.728447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:29:52.741560 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2c0e0-4077-41ad-9d3d-d9e48f311e39 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:29:52.745232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673ab17-9043-4f82-80ee-8536ff7f0636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:29:52.745890 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673ab17-9043-4f82-80ee-8536ff7f0636 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:29:52.746181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673ab17-9043-4f82-80ee-8536ff7f0636 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:29:52.746464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673ab17-9043-4f82-80ee-8536ff7f0636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:29:52.748313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-2673ab17-9043-4f82-80ee-8536ff7f0636 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:29:52.762776 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673ab17-9043-4f82-80ee-8536ff7f0636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:29:52.764267 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673ab17-9043-4f82-80ee-8536ff7f0636 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:29:52.777096 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1401/2787] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:29:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:29:55.869669 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=74 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:02.771718 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=6 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:05.813127 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33d444-e011-46c6-8a2d-c14e9f770dcd service cinder] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:05.815889 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-cb33d444-e011-46c6-8a2d-c14e9f770dcd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:05.906195 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33d444-e011-46c6-8a2d-c14e9f770dcd service cinder] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:05.912657 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf7e7a-71a4-45d4-9070-ecaba2e41cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:05.913966 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf7e7a-71a4-45d4-9070-ecaba2e41cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:05.914396 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf7e7a-71a4-45d4-9070-ecaba2e41cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:05.914820 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf7e7a-71a4-45d4-9070-ecaba2e41cd1 admin admin] PATH_INFO: `/v3/users/de859eecbef7498289f132a047e6e5f8` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:05.916613 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf7e7a-71a4-45d4-9070-ecaba2e41cd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de859eecbef7498289f132a047e6e5f8)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:05.919751 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf7e7a-71a4-45d4-9070-ecaba2e41cd1 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:05.979237 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a0cf7e7a-71a4-45d4-9070-ecaba2e41cd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de859eecbef7498289f132a047e6e5f8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:05.980300 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a0cf7e7a-71a4-45d4-9070-ecaba2e41cd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de859eecbef7498289f132a047e6e5f8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:05.993474 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a0cf7e7a-71a4-45d4-9070-ecaba2e41cd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de859eecbef7498289f132a047e6e5f8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:06.008960 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a0cf7e7a-71a4-45d4-9070-ecaba2e41cd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de859eecbef7498289f132a047e6e5f8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:06.009855 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-a0cf7e7a-71a4-45d4-9070-ecaba2e41cd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de859eecbef7498289f132a047e6e5f8'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:06.027700 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1387/2788] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:30:05 2022] DELETE /identity/v3/users/de859eecbef7498289f132a047e6e5f8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:06.219123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673ab17-9043-4f82-80ee-8536ff7f0636 service cinder] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:06.231420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673ab17-9043-4f82-80ee-8536ff7f0636 service cinder] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:06.295364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a54635-c8fa-431e-b5c5-e73277a89f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:06.296453 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a54635-c8fa-431e-b5c5-e73277a89f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:06.296854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a54635-c8fa-431e-b5c5-e73277a89f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:06.297264 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a54635-c8fa-431e-b5c5-e73277a89f07 admin admin] PATH_INFO: `/v3/projects/9a7262e606264aba8be1c0008e93c0d2` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:06.298724 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a54635-c8fa-431e-b5c5-e73277a89f07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a7262e606264aba8be1c0008e93c0d2)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:06.301429 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a54635-c8fa-431e-b5c5-e73277a89f07 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:06.360636 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-23a54635-c8fa-431e-b5c5-e73277a89f07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a7262e606264aba8be1c0008e93c0d2', 'request_id': 'req-23a54635-c8fa-431e-b5c5-e73277a89f07'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:06.368815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-23a54635-c8fa-431e-b5c5-e73277a89f07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a7262e606264aba8be1c0008e93c0d2', 'request_id': 'req-23a54635-c8fa-431e-b5c5-e73277a89f07'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:06.375481 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-23a54635-c8fa-431e-b5c5-e73277a89f07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a7262e606264aba8be1c0008e93c0d2', 'request_id': 'req-23a54635-c8fa-431e-b5c5-e73277a89f07'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:06.390172 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-23a54635-c8fa-431e-b5c5-e73277a89f07 admin admin] The token cache is being invalidate because project 9a7262e606264aba8be1c0008e93c0d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:30:06.390475 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-23a54635-c8fa-431e-b5c5-e73277a89f07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:06.391143 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-23a54635-c8fa-431e-b5c5-e73277a89f07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:06.393133 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1402/2789] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:30:06 2022] DELETE /identity/v3/projects/9a7262e606264aba8be1c0008e93c0d2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:06.399336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e0f96-e818-4f56-b95d-f9b1d5fb2f26 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:06.399776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e0f96-e818-4f56-b95d-f9b1d5fb2f26 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:06.400279 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e0f96-e818-4f56-b95d-f9b1d5fb2f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:06.421958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f30e0f96-e818-4f56-b95d-f9b1d5fb2f26 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:06.423199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f30e0f96-e818-4f56-b95d-f9b1d5fb2f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:06.488985 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1388/2790] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:06.498356 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848b985-4857-477a-aef7-d6115c04d848 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:06.498696 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848b985-4857-477a-aef7-d6115c04d848 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:06.499020 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848b985-4857-477a-aef7-d6115c04d848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:06.519667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-7848b985-4857-477a-aef7-d6115c04d848 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:06.520950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7848b985-4857-477a-aef7-d6115c04d848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:06.529651 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1403/2791] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:06.534885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e0f96-e818-4f56-b95d-f9b1d5fb2f26 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:06.545550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e0f96-e818-4f56-b95d-f9b1d5fb2f26 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:06.551107 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12394bfa-806a-46b5-992c-5355ac2f1106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:06.551832 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12394bfa-806a-46b5-992c-5355ac2f1106 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:06.552211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12394bfa-806a-46b5-992c-5355ac2f1106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:06.552538 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12394bfa-806a-46b5-992c-5355ac2f1106 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:06.553358 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12394bfa-806a-46b5-992c-5355ac2f1106 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:30:06.553780 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12394bfa-806a-46b5-992c-5355ac2f1106 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:06.555776 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12394bfa-806a-46b5-992c-5355ac2f1106 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:06.568507 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1389/2792] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:30:06.573174 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848b985-4857-477a-aef7-d6115c04d848 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:06.584091 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848b985-4857-477a-aef7-d6115c04d848 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:06.588908 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb0808-5999-4e3c-b490-bf293f94f49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:06.589587 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb0808-5999-4e3c-b490-bf293f94f49a admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:06.589875 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb0808-5999-4e3c-b490-bf293f94f49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:06.590169 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb0808-5999-4e3c-b490-bf293f94f49a admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:06.590865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb0808-5999-4e3c-b490-bf293f94f49a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:06.592893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb0808-5999-4e3c-b490-bf293f94f49a admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:06.641128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1404/2793] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:30:06 2022] POST /identity/v3/projects => generated 403 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:30:06.646934 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12394bfa-806a-46b5-992c-5355ac2f1106 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:06.657177 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12394bfa-806a-46b5-992c-5355ac2f1106 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:06.661120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:06.743798 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fc465-8c84-4a75-b980-c9320eec6378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:06.744881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fc465-8c84-4a75-b980-c9320eec6378 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:06.745384 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fc465-8c84-4a75-b980-c9320eec6378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:06.745868 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fc465-8c84-4a75-b980-c9320eec6378 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:06.746911 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fc465-8c84-4a75-b980-c9320eec6378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:06.750561 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fc465-8c84-4a75-b980-c9320eec6378 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:06.784504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1390/2794] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:30:06 2022] POST /identity/v3/users => generated 302 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:06.788537 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb0808-5999-4e3c-b490-bf293f94f49a admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:06.807498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb0808-5999-4e3c-b490-bf293f94f49a admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:06.812161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed7bc0-5044-424a-8722-29906123f7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:06.813076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed7bc0-5044-424a-8722-29906123f7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:06.813457 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed7bc0-5044-424a-8722-29906123f7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:06.813831 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed7bc0-5044-424a-8722-29906123f7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:06.814915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed7bc0-5044-424a-8722-29906123f7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:06.817420 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed7bc0-5044-424a-8722-29906123f7ae admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:06.837730 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1405/2795] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:06.839090 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fc465-8c84-4a75-b980-c9320eec6378 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:06.853354 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fc465-8c84-4a75-b980-c9320eec6378 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:06.858592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b58588-2fa4-4d89-be5d-bae26b73b890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:06.859712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b58588-2fa4-4d89-be5d-bae26b73b890 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:06.860003 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b58588-2fa4-4d89-be5d-bae26b73b890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:06.860316 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b58588-2fa4-4d89-be5d-bae26b73b890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:06.861060 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b58588-2fa4-4d89-be5d-bae26b73b890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:06.862693 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b58588-2fa4-4d89-be5d-bae26b73b890 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:06.873198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1391/2796] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:06.877507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed7bc0-5044-424a-8722-29906123f7ae admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:06.888884 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed7bc0-5044-424a-8722-29906123f7ae admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:06.894396 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134863a8-b6e6-4ab6-ab3f-d86e1040034f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:06.895292 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134863a8-b6e6-4ab6-ab3f-d86e1040034f admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:06.895579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134863a8-b6e6-4ab6-ab3f-d86e1040034f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:06.895859 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134863a8-b6e6-4ab6-ab3f-d86e1040034f admin admin] PATH_INFO: `/v3/projects/e6d12d8ca4a64423aca73d7cfc70d3d8/users/1782a3a9f9264fbfa9ec3ed67e72232a/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:06.910877 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134863a8-b6e6-4ab6-ab3f-d86e1040034f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d12d8ca4a64423aca73d7cfc70d3d8, user_id=1782a3a9f9264fbfa9ec3ed67e72232a, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:06.914182 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134863a8-b6e6-4ab6-ab3f-d86e1040034f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:06.927765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1406/2797] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:30:06 2022] PUT /identity/v3/projects/e6d12d8ca4a64423aca73d7cfc70d3d8/users/1782a3a9f9264fbfa9ec3ed67e72232a/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:30:06.933369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd2af3-5f4c-4276-926f-b2ade9fcf6ed None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:06.933663 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd2af3-5f4c-4276-926f-b2ade9fcf6ed None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:06.933947 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd2af3-5f4c-4276-926f-b2ade9fcf6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:06.952004 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-becd2af3-5f4c-4276-926f-b2ade9fcf6ed None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:06.953268 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-becd2af3-5f4c-4276-926f-b2ade9fcf6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:07.022629 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1392/2798] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:07.032277 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134863a8-b6e6-4ab6-ab3f-d86e1040034f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:07.034068 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-134863a8-b6e6-4ab6-ab3f-d86e1040034f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:07.065507 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:07.095931 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134863a8-b6e6-4ab6-ab3f-d86e1040034f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:07.099739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f19ca-a976-4832-a074-fda061dd83d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:07.101657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f19ca-a976-4832-a074-fda061dd83d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:07.101657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f19ca-a976-4832-a074-fda061dd83d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:07.101657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f19ca-a976-4832-a074-fda061dd83d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:07.110552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f19ca-a976-4832-a074-fda061dd83d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:07.112007 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f19ca-a976-4832-a074-fda061dd83d9 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:07.121847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1407/2799] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:30:07.808751 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6304-92b3-4284-bd5e-e5d0ea1d263a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:07.809051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6304-92b3-4284-bd5e-e5d0ea1d263a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:07.809343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6304-92b3-4284-bd5e-e5d0ea1d263a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:07.839305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-0dec6304-92b3-4284-bd5e-e5d0ea1d263a None None] MFA Rules not processed for user `1782a3a9f9264fbfa9ec3ed67e72232a`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:07.840684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-0dec6304-92b3-4284-bd5e-e5d0ea1d263a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:07.937251 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1393/2800] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:07.943989 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f19ca-a976-4832-a074-fda061dd83d9 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:07.960821 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f19ca-a976-4832-a074-fda061dd83d9 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:07.967940 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8aa92c-335c-4a2d-a6d5-b5b7ed5091ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:07.969026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aa92c-335c-4a2d-a6d5-b5b7ed5091ce admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:07.969484 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aa92c-335c-4a2d-a6d5-b5b7ed5091ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:07.969914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aa92c-335c-4a2d-a6d5-b5b7ed5091ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:07.971102 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8aa92c-335c-4a2d-a6d5-b5b7ed5091ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:07.973220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8aa92c-335c-4a2d-a6d5-b5b7ed5091ce admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:08.009815 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1408/2801] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:30:07 2022] POST /identity/v3/projects => generated 405 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:08.013904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6304-92b3-4284-bd5e-e5d0ea1d263a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:08.025509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6304-92b3-4284-bd5e-e5d0ea1d263a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:08.038916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:08.087952 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32794d5c-323c-4fed-96a7-08b253dc6cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:08.088668 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32794d5c-323c-4fed-96a7-08b253dc6cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:08.088955 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32794d5c-323c-4fed-96a7-08b253dc6cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:08.089233 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32794d5c-323c-4fed-96a7-08b253dc6cd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:08.089938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32794d5c-323c-4fed-96a7-08b253dc6cd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:08.091912 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32794d5c-323c-4fed-96a7-08b253dc6cd4 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:08.114093 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1394/2802] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:30:08 2022] POST /identity/v3/users => generated 309 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:30:08.118044 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8aa92c-335c-4a2d-a6d5-b5b7ed5091ce admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:08.130916 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8aa92c-335c-4a2d-a6d5-b5b7ed5091ce admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:08.140801 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed318293-75ce-4a47-b17c-a83253ca803d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:08.141498 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318293-75ce-4a47-b17c-a83253ca803d admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:08.141787 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318293-75ce-4a47-b17c-a83253ca803d admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:08.142068 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318293-75ce-4a47-b17c-a83253ca803d admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:08.142854 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed318293-75ce-4a47-b17c-a83253ca803d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:08.144565 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed318293-75ce-4a47-b17c-a83253ca803d admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:08.155938 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1409/2803] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:08.160509 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32794d5c-323c-4fed-96a7-08b253dc6cd4 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:08.170896 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32794d5c-323c-4fed-96a7-08b253dc6cd4 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:08.176858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec042b8-3ef6-4ffb-8b10-a88d1238b916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:08.177892 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec042b8-3ef6-4ffb-8b10-a88d1238b916 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:08.178217 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec042b8-3ef6-4ffb-8b10-a88d1238b916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:08.178548 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec042b8-3ef6-4ffb-8b10-a88d1238b916 admin admin] PATH_INFO: `/v3/projects/26906ecd033d4daca598474d8725b344/users/8e85788dd96d406798426d754b1a63ca/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:08.206817 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec042b8-3ef6-4ffb-8b10-a88d1238b916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26906ecd033d4daca598474d8725b344, user_id=8e85788dd96d406798426d754b1a63ca, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:08.209583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec042b8-3ef6-4ffb-8b10-a88d1238b916 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:08.225171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1395/2804] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:30:08 2022] PUT /identity/v3/projects/26906ecd033d4daca598474d8725b344/users/8e85788dd96d406798426d754b1a63ca/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:08.229523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed318293-75ce-4a47-b17c-a83253ca803d admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:08.241026 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed318293-75ce-4a47-b17c-a83253ca803d admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:08.305351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291253a7-9f94-45d4-b5dc-5de8b5c6e592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:08.305947 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291253a7-9f94-45d4-b5dc-5de8b5c6e592 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:08.306229 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291253a7-9f94-45d4-b5dc-5de8b5c6e592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:08.306505 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291253a7-9f94-45d4-b5dc-5de8b5c6e592 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:08.307238 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291253a7-9f94-45d4-b5dc-5de8b5c6e592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:08.308926 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291253a7-9f94-45d4-b5dc-5de8b5c6e592 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:08.322097 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1410/2805] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:08.326644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec042b8-3ef6-4ffb-8b10-a88d1238b916 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:08.339208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec042b8-3ef6-4ffb-8b10-a88d1238b916 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:08.344218 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9a3e0-3012-4090-8bb0-e283de1a1202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:08.345184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9a3e0-3012-4090-8bb0-e283de1a1202 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:08.345470 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9a3e0-3012-4090-8bb0-e283de1a1202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:08.345751 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9a3e0-3012-4090-8bb0-e283de1a1202 admin admin] PATH_INFO: `/v3/projects/26906ecd033d4daca598474d8725b344/users/8e85788dd96d406798426d754b1a63ca/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:08.348601 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9a3e0-3012-4090-8bb0-e283de1a1202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26906ecd033d4daca598474d8725b344, user_id=8e85788dd96d406798426d754b1a63ca, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:08.351363 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9a3e0-3012-4090-8bb0-e283de1a1202 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:08.362874 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1396/2806] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:30:08 2022] PUT /identity/v3/projects/26906ecd033d4daca598474d8725b344/users/8e85788dd96d406798426d754b1a63ca/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:09.087024 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac7354-ec85-4179-909b-e6f13b7cdd19 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:09.087635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac7354-ec85-4179-909b-e6f13b7cdd19 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:09.088015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac7354-ec85-4179-909b-e6f13b7cdd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:09.132892 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-21ac7354-ec85-4179-909b-e6f13b7cdd19 None None] MFA Rules not processed for user `8e85788dd96d406798426d754b1a63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:09.134171 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-21ac7354-ec85-4179-909b-e6f13b7cdd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:09.224202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1411/2807] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:09 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:10.361592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9a3e0-3012-4090-8bb0-e283de1a1202 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:10.432758 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9a3e0-3012-4090-8bb0-e283de1a1202 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:10.439179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:10.440147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:10.440437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:10.440714 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] PATH_INFO: `/v3/users/1782a3a9f9264fbfa9ec3ed67e72232a` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:10.442122 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1782a3a9f9264fbfa9ec3ed67e72232a)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:10.444086 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:10.492857 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1782a3a9f9264fbfa9ec3ed67e72232a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:10.493549 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1782a3a9f9264fbfa9ec3ed67e72232a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:10.500927 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1782a3a9f9264fbfa9ec3ed67e72232a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:10.511346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:10.513914 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1782a3a9f9264fbfa9ec3ed67e72232a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:10.514916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1782a3a9f9264fbfa9ec3ed67e72232a'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:10.529867 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1397/2808] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:30:10 2022] DELETE /identity/v3/users/1782a3a9f9264fbfa9ec3ed67e72232a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:10.711122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac7354-ec85-4179-909b-e6f13b7cdd19 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:10.723012 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac7354-ec85-4179-909b-e6f13b7cdd19 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:10.737888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:10.788952 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:10.789855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:10.790141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:10.790435 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] PATH_INFO: `/v3/projects/e6d12d8ca4a64423aca73d7cfc70d3d8` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:10.791702 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d12d8ca4a64423aca73d7cfc70d3d8)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:10.793703 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:10.848545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d12d8ca4a64423aca73d7cfc70d3d8', 'request_id': 'req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:10.855800 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d12d8ca4a64423aca73d7cfc70d3d8', 'request_id': 'req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:10.863219 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d12d8ca4a64423aca73d7cfc70d3d8', 'request_id': 'req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:10.878045 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] The token cache is being invalidate because project e6d12d8ca4a64423aca73d7cfc70d3d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:30:10.878340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:10.879029 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:10.881189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1412/2809] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:30:10 2022] DELETE /identity/v3/projects/e6d12d8ca4a64423aca73d7cfc70d3d8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:10.885343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:10.886926 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:10.901161 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe7400-ad0c-4862-a080-513065bcfea6 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:10.966779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3be5f3-6eaf-4f1b-9124-3940e086953a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:10.967712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3be5f3-6eaf-4f1b-9124-3940e086953a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:10.967997 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3be5f3-6eaf-4f1b-9124-3940e086953a admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:10.968313 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3be5f3-6eaf-4f1b-9124-3940e086953a admin admin] PATH_INFO: `/v3/users/8e85788dd96d406798426d754b1a63ca` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:10.969584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3be5f3-6eaf-4f1b-9124-3940e086953a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e85788dd96d406798426d754b1a63ca)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:10.972158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3be5f3-6eaf-4f1b-9124-3940e086953a admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:11.035031 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-dd3be5f3-6eaf-4f1b-9124-3940e086953a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e85788dd96d406798426d754b1a63ca'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:11.036149 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-dd3be5f3-6eaf-4f1b-9124-3940e086953a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e85788dd96d406798426d754b1a63ca'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:11.044109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-dd3be5f3-6eaf-4f1b-9124-3940e086953a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e85788dd96d406798426d754b1a63ca'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:11.063757 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-dd3be5f3-6eaf-4f1b-9124-3940e086953a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e85788dd96d406798426d754b1a63ca'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:11.065313 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-dd3be5f3-6eaf-4f1b-9124-3940e086953a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e85788dd96d406798426d754b1a63ca'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:11.079789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1398/2810] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:30:10 2022] DELETE /identity/v3/users/8e85788dd96d406798426d754b1a63ca => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:11.297089 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:11.299023 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:11.315410 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acd2f0-cc2d-49eb-9cc6-f6a082bd3e13 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:11.407003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fceda84-b63f-468c-ae91-ada8250d6522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:11.407929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fceda84-b63f-468c-ae91-ada8250d6522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:11.408237 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fceda84-b63f-468c-ae91-ada8250d6522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:11.409067 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fceda84-b63f-468c-ae91-ada8250d6522 admin admin] PATH_INFO: `/v3/projects/26906ecd033d4daca598474d8725b344` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:11.410381 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fceda84-b63f-468c-ae91-ada8250d6522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26906ecd033d4daca598474d8725b344)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:11.412397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fceda84-b63f-468c-ae91-ada8250d6522 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:11.484804 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-2fceda84-b63f-468c-ae91-ada8250d6522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26906ecd033d4daca598474d8725b344', 'request_id': 'req-2fceda84-b63f-468c-ae91-ada8250d6522'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:11.486474 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:11.496364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-2fceda84-b63f-468c-ae91-ada8250d6522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26906ecd033d4daca598474d8725b344', 'request_id': 'req-2fceda84-b63f-468c-ae91-ada8250d6522'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:11.504448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-2fceda84-b63f-468c-ae91-ada8250d6522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26906ecd033d4daca598474d8725b344', 'request_id': 'req-2fceda84-b63f-468c-ae91-ada8250d6522'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:11.522157 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-2fceda84-b63f-468c-ae91-ada8250d6522 admin admin] The token cache is being invalidate because project 26906ecd033d4daca598474d8725b344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:30:11.522583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-2fceda84-b63f-468c-ae91-ada8250d6522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:11.523404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-2fceda84-b63f-468c-ae91-ada8250d6522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:11.525448 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1413/2811] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:30:11 2022] DELETE /identity/v3/projects/26906ecd033d4daca598474d8725b344 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:11.531804 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476baca6-e7bc-4ca9-b42c-af82c38fa63b None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:11.532525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476baca6-e7bc-4ca9-b42c-af82c38fa63b None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:11.532525 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476baca6-e7bc-4ca9-b42c-af82c38fa63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:11.556679 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-476baca6-e7bc-4ca9-b42c-af82c38fa63b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:11.557916 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-476baca6-e7bc-4ca9-b42c-af82c38fa63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:11.630346 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1399/2812] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:11.639812 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de844d-800f-4b84-92d5-fe4b06a82cce None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:11.640170 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de844d-800f-4b84-92d5-fe4b06a82cce None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:11.640459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de844d-800f-4b84-92d5-fe4b06a82cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:11.661482 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-68de844d-800f-4b84-92d5-fe4b06a82cce None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:11.662738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-68de844d-800f-4b84-92d5-fe4b06a82cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:11.671919 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1414/2813] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:11.677073 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476baca6-e7bc-4ca9-b42c-af82c38fa63b None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:11.687977 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476baca6-e7bc-4ca9-b42c-af82c38fa63b None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:11.694067 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595df5d-acff-4966-8d87-924f66e4e432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:11.694759 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595df5d-acff-4966-8d87-924f66e4e432 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:11.695050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595df5d-acff-4966-8d87-924f66e4e432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:11.695327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595df5d-acff-4966-8d87-924f66e4e432 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:11.696111 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595df5d-acff-4966-8d87-924f66e4e432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:30:11.696426 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595df5d-acff-4966-8d87-924f66e4e432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:11.698079 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595df5d-acff-4966-8d87-924f66e4e432 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:11.699918 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:11.713260 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1400/2814] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:30:11.718207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de844d-800f-4b84-92d5-fe4b06a82cce None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:11.729198 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de844d-800f-4b84-92d5-fe4b06a82cce None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:11.734076 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4401532-98f8-4abe-89a6-693ff02c09db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:11.734755 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4401532-98f8-4abe-89a6-693ff02c09db admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:11.735046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4401532-98f8-4abe-89a6-693ff02c09db admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:11.735325 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4401532-98f8-4abe-89a6-693ff02c09db admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:11.736034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4401532-98f8-4abe-89a6-693ff02c09db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:11.738620 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4401532-98f8-4abe-89a6-693ff02c09db admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:11.770876 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1415/2815] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:30:11 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:11.775924 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595df5d-acff-4966-8d87-924f66e4e432 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:11.787677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595df5d-acff-4966-8d87-924f66e4e432 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:11.854400 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d1ad0-fbf9-4d37-9546-bc905a1db105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:11.855080 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d1ad0-fbf9-4d37-9546-bc905a1db105 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:11.855369 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d1ad0-fbf9-4d37-9546-bc905a1db105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:11.855648 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d1ad0-fbf9-4d37-9546-bc905a1db105 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:11.856404 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d1ad0-fbf9-4d37-9546-bc905a1db105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:11.858345 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d1ad0-fbf9-4d37-9546-bc905a1db105 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:11.888779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1401/2816] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:30:11 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:11.893242 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4401532-98f8-4abe-89a6-693ff02c09db admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:11.904611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4401532-98f8-4abe-89a6-693ff02c09db admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:11.909695 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22c1e2-ae05-4532-9aa2-3cc43f9b135a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:11.910407 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22c1e2-ae05-4532-9aa2-3cc43f9b135a admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:11.910646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22c1e2-ae05-4532-9aa2-3cc43f9b135a admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:11.910923 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22c1e2-ae05-4532-9aa2-3cc43f9b135a admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:11.911847 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22c1e2-ae05-4532-9aa2-3cc43f9b135a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:11.913434 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22c1e2-ae05-4532-9aa2-3cc43f9b135a admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:11.924950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1416/2817] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:30:11.929074 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d1ad0-fbf9-4d37-9546-bc905a1db105 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:11.940732 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d1ad0-fbf9-4d37-9546-bc905a1db105 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:11.945559 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc8326-bfb1-435f-b23c-c9db9ea92fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:11.946238 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc8326-bfb1-435f-b23c-c9db9ea92fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:11.946546 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc8326-bfb1-435f-b23c-c9db9ea92fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:11.946828 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc8326-bfb1-435f-b23c-c9db9ea92fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:11.947589 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc8326-bfb1-435f-b23c-c9db9ea92fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:11.949240 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc8326-bfb1-435f-b23c-c9db9ea92fb8 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:11.960327 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1402/2818] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:11.965011 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22c1e2-ae05-4532-9aa2-3cc43f9b135a admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:11.975605 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22c1e2-ae05-4532-9aa2-3cc43f9b135a admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:11.980936 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf1732-1ea9-44cf-9b89-e9193b770883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:11.981835 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf1732-1ea9-44cf-9b89-e9193b770883 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:11.982119 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf1732-1ea9-44cf-9b89-e9193b770883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:11.982428 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf1732-1ea9-44cf-9b89-e9193b770883 admin admin] PATH_INFO: `/v3/projects/5793f8afb5bf47caa22cf67f32858343/users/2ed314794e874ab383475b5e7e8d523b/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:11.998768 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf1732-1ea9-44cf-9b89-e9193b770883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5793f8afb5bf47caa22cf67f32858343, user_id=2ed314794e874ab383475b5e7e8d523b, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:12.001622 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf1732-1ea9-44cf-9b89-e9193b770883 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:12.016145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1417/2819] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:30:11 2022] PUT /identity/v3/projects/5793f8afb5bf47caa22cf67f32858343/users/2ed314794e874ab383475b5e7e8d523b/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:12.027740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78036e-570c-4a28-98a6-2447113b830a None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:12.028054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78036e-570c-4a28-98a6-2447113b830a None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:12.028372 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78036e-570c-4a28-98a6-2447113b830a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:12.056978 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-6a78036e-570c-4a28-98a6-2447113b830a None None] MFA Rules not processed for user `2ed314794e874ab383475b5e7e8d523b`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:12.058208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-6a78036e-570c-4a28-98a6-2447113b830a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:12.128501 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1403/2820] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:12.133384 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf1732-1ea9-44cf-9b89-e9193b770883 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:12.197650 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf1732-1ea9-44cf-9b89-e9193b770883 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:12.202661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae2b69-0d4c-48a6-a1f8-951d3ea48f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:12.203336 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae2b69-0d4c-48a6-a1f8-951d3ea48f91 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:12.203623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae2b69-0d4c-48a6-a1f8-951d3ea48f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:12.203902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae2b69-0d4c-48a6-a1f8-951d3ea48f91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:12.204632 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae2b69-0d4c-48a6-a1f8-951d3ea48f91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:12.206607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae2b69-0d4c-48a6-a1f8-951d3ea48f91 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:12.243860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1418/2821] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:30:12 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:12.248765 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78036e-570c-4a28-98a6-2447113b830a None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:12.259813 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78036e-570c-4a28-98a6-2447113b830a None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:12.322075 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57ec91-9fed-4d18-8a1e-9065ee3bd1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:12.322858 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57ec91-9fed-4d18-8a1e-9065ee3bd1bb admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:12.323283 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57ec91-9fed-4d18-8a1e-9065ee3bd1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:12.323634 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57ec91-9fed-4d18-8a1e-9065ee3bd1bb admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:12.324618 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57ec91-9fed-4d18-8a1e-9065ee3bd1bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:12.327109 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57ec91-9fed-4d18-8a1e-9065ee3bd1bb admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:12.335579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:12.349096 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1404/2822] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:30:12 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:12.353706 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae2b69-0d4c-48a6-a1f8-951d3ea48f91 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:12.367031 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae2b69-0d4c-48a6-a1f8-951d3ea48f91 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:12.372034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc781b31-6a23-48eb-99a9-981aa45ef3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:12.372888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc781b31-6a23-48eb-99a9-981aa45ef3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:12.373207 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc781b31-6a23-48eb-99a9-981aa45ef3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:12.373643 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc781b31-6a23-48eb-99a9-981aa45ef3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:12.374436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc781b31-6a23-48eb-99a9-981aa45ef3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:12.376618 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc781b31-6a23-48eb-99a9-981aa45ef3a1 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:12.390169 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1419/2823] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:12.395318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57ec91-9fed-4d18-8a1e-9065ee3bd1bb admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:12.406029 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57ec91-9fed-4d18-8a1e-9065ee3bd1bb admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:12.411902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09b872-c0a7-45b2-870f-27c2da9e696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:12.413062 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09b872-c0a7-45b2-870f-27c2da9e696e admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:12.413438 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09b872-c0a7-45b2-870f-27c2da9e696e admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:12.413790 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09b872-c0a7-45b2-870f-27c2da9e696e admin admin] PATH_INFO: `/v3/projects/2a43240a1f7a4abc974491227155cd80/users/f5ef856fe39a44329cb43df38add575b/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:12.430665 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09b872-c0a7-45b2-870f-27c2da9e696e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a43240a1f7a4abc974491227155cd80, user_id=f5ef856fe39a44329cb43df38add575b, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:12.433398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09b872-c0a7-45b2-870f-27c2da9e696e admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:12.447419 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1405/2824] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:30:12 2022] PUT /identity/v3/projects/2a43240a1f7a4abc974491227155cd80/users/f5ef856fe39a44329cb43df38add575b/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:30:12.451639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc781b31-6a23-48eb-99a9-981aa45ef3a1 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:12.461902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc781b31-6a23-48eb-99a9-981aa45ef3a1 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:12.509200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:12.528569 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2de10a-e612-4283-bbfc-19dffcdccb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:12.529249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2de10a-e612-4283-bbfc-19dffcdccb41 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:12.529563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2de10a-e612-4283-bbfc-19dffcdccb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:12.529866 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2de10a-e612-4283-bbfc-19dffcdccb41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:12.530594 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2de10a-e612-4283-bbfc-19dffcdccb41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:12.532238 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2de10a-e612-4283-bbfc-19dffcdccb41 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:12.544278 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1420/2825] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:12.548615 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09b872-c0a7-45b2-870f-27c2da9e696e admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:12.560500 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09b872-c0a7-45b2-870f-27c2da9e696e admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:12.565410 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506332d-7635-4893-923e-db0e8b0ec725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:12.566550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506332d-7635-4893-923e-db0e8b0ec725 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:12.566975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506332d-7635-4893-923e-db0e8b0ec725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:12.567442 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506332d-7635-4893-923e-db0e8b0ec725 admin admin] PATH_INFO: `/v3/projects/2a43240a1f7a4abc974491227155cd80/users/f5ef856fe39a44329cb43df38add575b/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:12.570791 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506332d-7635-4893-923e-db0e8b0ec725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a43240a1f7a4abc974491227155cd80, user_id=f5ef856fe39a44329cb43df38add575b, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:12.576142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506332d-7635-4893-923e-db0e8b0ec725 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:12.594748 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1406/2826] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:30:12 2022] PUT /identity/v3/projects/2a43240a1f7a4abc974491227155cd80/users/f5ef856fe39a44329cb43df38add575b/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:12.605852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71c2b7-850f-4f16-90e9-0fc0d792c2bd None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:12.606123 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71c2b7-850f-4f16-90e9-0fc0d792c2bd None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:12.606415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71c2b7-850f-4f16-90e9-0fc0d792c2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:12.634712 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-1a71c2b7-850f-4f16-90e9-0fc0d792c2bd None None] MFA Rules not processed for user `f5ef856fe39a44329cb43df38add575b`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:12.635949 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1a71c2b7-850f-4f16-90e9-0fc0d792c2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:12.715346 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1421/2827] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:12.728248 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506332d-7635-4893-923e-db0e8b0ec725 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:12.730415 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-7506332d-7635-4893-923e-db0e8b0ec725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:12.818207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506332d-7635-4893-923e-db0e8b0ec725 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:12.823873 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782ab57-02b1-4947-94cc-c3243e6ad31a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:12.824877 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782ab57-02b1-4947-94cc-c3243e6ad31a service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:12.825304 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782ab57-02b1-4947-94cc-c3243e6ad31a service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:12.825718 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782ab57-02b1-4947-94cc-c3243e6ad31a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:12.840965 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782ab57-02b1-4947-94cc-c3243e6ad31a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:12.843411 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782ab57-02b1-4947-94cc-c3243e6ad31a service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:12.859379 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1407/2828] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 07:30:13.320144 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71c2b7-850f-4f16-90e9-0fc0d792c2bd None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:13.384332 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71c2b7-850f-4f16-90e9-0fc0d792c2bd None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:13.389106 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:13.390380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:13.390380 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:13.390561 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] PATH_INFO: `/v3/users/2ed314794e874ab383475b5e7e8d523b` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:13.391749 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed314794e874ab383475b5e7e8d523b)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:13.394295 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:13.448762 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed314794e874ab383475b5e7e8d523b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:13.457763 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed314794e874ab383475b5e7e8d523b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:13.458419 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed314794e874ab383475b5e7e8d523b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:13.473361 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed314794e874ab383475b5e7e8d523b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:13.474088 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed314794e874ab383475b5e7e8d523b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:13.475301 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:13.490069 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1422/2829] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:30:13 2022] DELETE /identity/v3/users/2ed314794e874ab383475b5e7e8d523b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:13.495679 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c8220-b92b-4b28-bcc9-6db083baa2d8 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:13.495967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c8220-b92b-4b28-bcc9-6db083baa2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:13.496269 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c8220-b92b-4b28-bcc9-6db083baa2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:13.515257 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-9b8c8220-b92b-4b28-bcc9-6db083baa2d8 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:13.516562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9b8c8220-b92b-4b28-bcc9-6db083baa2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:13.577188 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1408/2830] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:30:13.587295 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:13.588769 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:13.647071 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269c7c0-6899-49cb-aa00-74502d83c3e8 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:13.650739 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d6b62-dbe4-429f-a150-ad3278049bd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:13.651368 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d6b62-dbe4-429f-a150-ad3278049bd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:13.651650 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d6b62-dbe4-429f-a150-ad3278049bd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:13.651924 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d6b62-dbe4-429f-a150-ad3278049bd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:13.661296 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d6b62-dbe4-429f-a150-ad3278049bd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:13.663218 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d6b62-dbe4-429f-a150-ad3278049bd8 service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:13.673034 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1423/2831] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:30:13.959880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c8220-b92b-4b28-bcc9-6db083baa2d8 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:13.961929 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9b8c8220-b92b-4b28-bcc9-6db083baa2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:13.980142 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c8220-b92b-4b28-bcc9-6db083baa2d8 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:13.984881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:13.985767 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:13.986050 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:13.986336 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] PATH_INFO: `/v3/projects/5793f8afb5bf47caa22cf67f32858343` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:13.988222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] RBAC: Authorizing `identity:delete_project(project_id=5793f8afb5bf47caa22cf67f32858343)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:13.990135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:14.050550 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5793f8afb5bf47caa22cf67f32858343', 'request_id': 'req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.053659 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:14.060861 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5793f8afb5bf47caa22cf67f32858343', 'request_id': 'req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.070717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5793f8afb5bf47caa22cf67f32858343', 'request_id': 'req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.082224 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] The token cache is being invalidate because project 5793f8afb5bf47caa22cf67f32858343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:30:14.082514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.083186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.084797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1409/2832] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:30:13 2022] DELETE /identity/v3/projects/5793f8afb5bf47caa22cf67f32858343 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:14.089666 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d6b62-dbe4-429f-a150-ad3278049bd8 service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:14.101228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d6b62-dbe4-429f-a150-ad3278049bd8 service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:14.168864 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163a3f2-1bd0-4195-9bb1-9c4809f2c8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:14.169757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163a3f2-1bd0-4195-9bb1-9c4809f2c8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:14.170042 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163a3f2-1bd0-4195-9bb1-9c4809f2c8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:14.170321 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163a3f2-1bd0-4195-9bb1-9c4809f2c8ce admin admin] PATH_INFO: `/v3/users/f5ef856fe39a44329cb43df38add575b` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:14.171545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163a3f2-1bd0-4195-9bb1-9c4809f2c8ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ef856fe39a44329cb43df38add575b)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:14.173583 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163a3f2-1bd0-4195-9bb1-9c4809f2c8ce admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:14.233508 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1163a3f2-1bd0-4195-9bb1-9c4809f2c8ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ef856fe39a44329cb43df38add575b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.246665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1163a3f2-1bd0-4195-9bb1-9c4809f2c8ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ef856fe39a44329cb43df38add575b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.247306 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1163a3f2-1bd0-4195-9bb1-9c4809f2c8ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ef856fe39a44329cb43df38add575b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.262436 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1163a3f2-1bd0-4195-9bb1-9c4809f2c8ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ef856fe39a44329cb43df38add575b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.263256 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1163a3f2-1bd0-4195-9bb1-9c4809f2c8ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ef856fe39a44329cb43df38add575b'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.279095 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1424/2833] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:30:14 2022] DELETE /identity/v3/users/f5ef856fe39a44329cb43df38add575b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:14.585044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:14.586593 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:14.601432 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ec287-9ee0-4c01-8f22-9246bfa54fda admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:14.666100 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f449608-32fb-436f-b2dd-2b87c461cbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:14.666959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f449608-32fb-436f-b2dd-2b87c461cbdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:14.667305 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f449608-32fb-436f-b2dd-2b87c461cbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:14.667611 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f449608-32fb-436f-b2dd-2b87c461cbdc admin admin] PATH_INFO: `/v3/projects/2a43240a1f7a4abc974491227155cd80` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:14.668931 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f449608-32fb-436f-b2dd-2b87c461cbdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a43240a1f7a4abc974491227155cd80)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:14.670967 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f449608-32fb-436f-b2dd-2b87c461cbdc admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:14.720598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f449608-32fb-436f-b2dd-2b87c461cbdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a43240a1f7a4abc974491227155cd80', 'request_id': 'req-8f449608-32fb-436f-b2dd-2b87c461cbdc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.728394 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f449608-32fb-436f-b2dd-2b87c461cbdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a43240a1f7a4abc974491227155cd80', 'request_id': 'req-8f449608-32fb-436f-b2dd-2b87c461cbdc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.734656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f449608-32fb-436f-b2dd-2b87c461cbdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a43240a1f7a4abc974491227155cd80', 'request_id': 'req-8f449608-32fb-436f-b2dd-2b87c461cbdc'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.746028 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f449608-32fb-436f-b2dd-2b87c461cbdc admin admin] The token cache is being invalidate because project 2a43240a1f7a4abc974491227155cd80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80117) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:30:14.746318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f449608-32fb-436f-b2dd-2b87c461cbdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.746957 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8f449608-32fb-436f-b2dd-2b87c461cbdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:14.748692 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1410/2834] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:30:14 2022] DELETE /identity/v3/projects/2a43240a1f7a4abc974491227155cd80 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:14.753993 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcefc0-010e-4c1a-afd8-2d719d734d45 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:14.754464 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcefc0-010e-4c1a-afd8-2d719d734d45 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:14.754588 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcefc0-010e-4c1a-afd8-2d719d734d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:14.772335 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-6cbcefc0-010e-4c1a-afd8-2d719d734d45 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:14.773613 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-6cbcefc0-010e-4c1a-afd8-2d719d734d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:14.780600 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:14.844978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1425/2835] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:14.853865 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785963d-8e85-47a5-b346-5d58649645f5 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:14.854179 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785963d-8e85-47a5-b346-5d58649645f5 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:14.854464 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785963d-8e85-47a5-b346-5d58649645f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:14.881883 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-4785963d-8e85-47a5-b346-5d58649645f5 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:14.883814 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-4785963d-8e85-47a5-b346-5d58649645f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:14.896885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1411/2836] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:30:14.901233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcefc0-010e-4c1a-afd8-2d719d734d45 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:14.912006 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcefc0-010e-4c1a-afd8-2d719d734d45 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:14.918070 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b10a94-536b-48f0-8dbb-5339360ba136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:14.921738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b10a94-536b-48f0-8dbb-5339360ba136 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:14.921738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b10a94-536b-48f0-8dbb-5339360ba136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:14.921738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b10a94-536b-48f0-8dbb-5339360ba136 admin admin] PATH_INFO: `/v3/domains` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:14.921738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b10a94-536b-48f0-8dbb-5339360ba136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80118) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:30:14.921738 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b10a94-536b-48f0-8dbb-5339360ba136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:14.922351 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b10a94-536b-48f0-8dbb-5339360ba136 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:14.936279 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1426/2837] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:30:14.941172 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785963d-8e85-47a5-b346-5d58649645f5 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:14.950594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785963d-8e85-47a5-b346-5d58649645f5 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:14.955294 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b9001-fc13-4663-bf7f-cad6f5bbc389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:14.955990 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b9001-fc13-4663-bf7f-cad6f5bbc389 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:14.956307 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b9001-fc13-4663-bf7f-cad6f5bbc389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:14.956768 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b9001-fc13-4663-bf7f-cad6f5bbc389 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:14.957466 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b9001-fc13-4663-bf7f-cad6f5bbc389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:14.959374 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b9001-fc13-4663-bf7f-cad6f5bbc389 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:14.997022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1412/2838] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:30:14 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:15.001189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b10a94-536b-48f0-8dbb-5339360ba136 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:15.013032 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b10a94-536b-48f0-8dbb-5339360ba136 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:15.072991 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022af07-9c6f-4f67-99fc-31f171158862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:15.073736 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022af07-9c6f-4f67-99fc-31f171158862 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.073985 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022af07-9c6f-4f67-99fc-31f171158862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.074148 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022af07-9c6f-4f67-99fc-31f171158862 admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.074850 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022af07-9c6f-4f67-99fc-31f171158862 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:15.076832 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022af07-9c6f-4f67-99fc-31f171158862 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:15.097552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1427/2839] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:30:15 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:15.102084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b9001-fc13-4663-bf7f-cad6f5bbc389 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:15.114186 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b9001-fc13-4663-bf7f-cad6f5bbc389 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:15.118973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61894952-67cd-47b1-b2eb-7ccee15834d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:15.119643 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61894952-67cd-47b1-b2eb-7ccee15834d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.119936 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61894952-67cd-47b1-b2eb-7ccee15834d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.120255 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61894952-67cd-47b1-b2eb-7ccee15834d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.121035 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61894952-67cd-47b1-b2eb-7ccee15834d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:15.122684 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61894952-67cd-47b1-b2eb-7ccee15834d4 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:15.134381 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1413/2840] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:15.138582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022af07-9c6f-4f67-99fc-31f171158862 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:15.148966 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022af07-9c6f-4f67-99fc-31f171158862 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:15.153563 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82955934-d808-4dbc-b331-1b9611d5aef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:15.154233 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82955934-d808-4dbc-b331-1b9611d5aef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.154518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82955934-d808-4dbc-b331-1b9611d5aef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.154796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82955934-d808-4dbc-b331-1b9611d5aef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.155518 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82955934-d808-4dbc-b331-1b9611d5aef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:15.157166 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82955934-d808-4dbc-b331-1b9611d5aef7 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:15.167789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1428/2841] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:15.172241 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61894952-67cd-47b1-b2eb-7ccee15834d4 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:15.183130 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61894952-67cd-47b1-b2eb-7ccee15834d4 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:15.187844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e647f-7af0-471d-b7d1-4c73d2644945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:15.188751 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e647f-7af0-471d-b7d1-4c73d2644945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.189044 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e647f-7af0-471d-b7d1-4c73d2644945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.189449 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e647f-7af0-471d-b7d1-4c73d2644945 admin admin] PATH_INFO: `/v3/projects/453e045c5b2544d59a5b42256a194c14/users/ccdfa3b931614da0b202b1ce00824367/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.205865 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e647f-7af0-471d-b7d1-4c73d2644945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=453e045c5b2544d59a5b42256a194c14, user_id=ccdfa3b931614da0b202b1ce00824367, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:15.208704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e647f-7af0-471d-b7d1-4c73d2644945 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:15.223227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1414/2842] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:30:15 2022] PUT /identity/v3/projects/453e045c5b2544d59a5b42256a194c14/users/ccdfa3b931614da0b202b1ce00824367/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:15.234153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420510c-75c8-42cf-9b9b-58a2676ac4ac None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.234447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420510c-75c8-42cf-9b9b-58a2676ac4ac None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.234731 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420510c-75c8-42cf-9b9b-58a2676ac4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.263539 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-3420510c-75c8-42cf-9b9b-58a2676ac4ac None None] MFA Rules not processed for user `ccdfa3b931614da0b202b1ce00824367`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:15.264856 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3420510c-75c8-42cf-9b9b-58a2676ac4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:15.331967 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1429/2843] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:15.336504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e647f-7af0-471d-b7d1-4c73d2644945 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:15.345389 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:15.406343 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e647f-7af0-471d-b7d1-4c73d2644945 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:15.411333 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2ba41-5246-4c31-b622-60a03be29852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:15.412005 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2ba41-5246-4c31-b622-60a03be29852 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.412318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2ba41-5246-4c31-b622-60a03be29852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.412596 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2ba41-5246-4c31-b622-60a03be29852 admin admin] PATH_INFO: `/v3/projects` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.413289 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2ba41-5246-4c31-b622-60a03be29852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:15.415296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2ba41-5246-4c31-b622-60a03be29852 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:15.443329 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1415/2844] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:30:15 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:15.447690 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420510c-75c8-42cf-9b9b-58a2676ac4ac None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:15.459855 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420510c-75c8-42cf-9b9b-58a2676ac4ac None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:15.463617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:15.524656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23cf1c-8506-4b17-ba0f-02331329d4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:15.525358 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23cf1c-8506-4b17-ba0f-02331329d4be admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.525645 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23cf1c-8506-4b17-ba0f-02331329d4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.525925 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23cf1c-8506-4b17-ba0f-02331329d4be admin admin] PATH_INFO: `/v3/users` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.526635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23cf1c-8506-4b17-ba0f-02331329d4be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:15.528617 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23cf1c-8506-4b17-ba0f-02331329d4be admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:15.548984 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1430/2845] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:30:15 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:15.553059 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2ba41-5246-4c31-b622-60a03be29852 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:15.563986 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2ba41-5246-4c31-b622-60a03be29852 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:15.569475 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ff348-0fc3-4dc7-adff-a8b5b7b17b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:15.570165 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ff348-0fc3-4dc7-adff-a8b5b7b17b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.570451 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ff348-0fc3-4dc7-adff-a8b5b7b17b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.570754 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ff348-0fc3-4dc7-adff-a8b5b7b17b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.571493 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ff348-0fc3-4dc7-adff-a8b5b7b17b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:15.573151 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ff348-0fc3-4dc7-adff-a8b5b7b17b73 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:15.585022 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1416/2846] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 07:30:15.590150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23cf1c-8506-4b17-ba0f-02331329d4be admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:15.600596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23cf1c-8506-4b17-ba0f-02331329d4be admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:15.605331 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfefd6-bcbc-4176-b71d-b210c610a4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:15.606255 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfefd6-bcbc-4176-b71d-b210c610a4f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.606550 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfefd6-bcbc-4176-b71d-b210c610a4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.606951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfefd6-bcbc-4176-b71d-b210c610a4f4 admin admin] PATH_INFO: `/v3/projects/e36a328cade748a8b22fe186c7eac6d9/users/8f5dc9a232374ab5bc108e9d106e0353/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.623295 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfefd6-bcbc-4176-b71d-b210c610a4f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e36a328cade748a8b22fe186c7eac6d9, user_id=8f5dc9a232374ab5bc108e9d106e0353, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:15.626055 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfefd6-bcbc-4176-b71d-b210c610a4f4 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:15.640385 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1431/2847] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:30:15 2022] PUT /identity/v3/projects/e36a328cade748a8b22fe186c7eac6d9/users/8f5dc9a232374ab5bc108e9d106e0353/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:15.644514 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ff348-0fc3-4dc7-adff-a8b5b7b17b73 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:15.654627 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ff348-0fc3-4dc7-adff-a8b5b7b17b73 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:15.712887 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ea052-a7d0-4850-b639-f167315e6740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:15.713481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ea052-a7d0-4850-b639-f167315e6740 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.713769 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ea052-a7d0-4850-b639-f167315e6740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.714051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ea052-a7d0-4850-b639-f167315e6740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.714784 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ea052-a7d0-4850-b639-f167315e6740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:15.716447 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ea052-a7d0-4850-b639-f167315e6740 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:15.727001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1417/2848] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:15.732193 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfefd6-bcbc-4176-b71d-b210c610a4f4 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:15.744136 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfefd6-bcbc-4176-b71d-b210c610a4f4 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:15.748888 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c62a9-498e-43b6-a849-8cb3debc4371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:15.749778 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c62a9-498e-43b6-a849-8cb3debc4371 admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.750064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c62a9-498e-43b6-a849-8cb3debc4371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.750375 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c62a9-498e-43b6-a849-8cb3debc4371 admin admin] PATH_INFO: `/v3/projects/e36a328cade748a8b22fe186c7eac6d9/users/8f5dc9a232374ab5bc108e9d106e0353/roles/c625cb8707d040dfbdcf8f218c0a8287` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.753128 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c62a9-498e-43b6-a849-8cb3debc4371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e36a328cade748a8b22fe186c7eac6d9, user_id=8f5dc9a232374ab5bc108e9d106e0353, role_id=c625cb8707d040dfbdcf8f218c0a8287)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:15.755611 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c62a9-498e-43b6-a849-8cb3debc4371 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:15.766280 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1432/2849] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:30:15 2022] PUT /identity/v3/projects/e36a328cade748a8b22fe186c7eac6d9/users/8f5dc9a232374ab5bc108e9d106e0353/roles/c625cb8707d040dfbdcf8f218c0a8287 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:15.777562 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca848e-9dc1-4cbd-be7a-bcfaa4bfd1c7 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.777902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca848e-9dc1-4cbd-be7a-bcfaa4bfd1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.778169 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca848e-9dc1-4cbd-be7a-bcfaa4bfd1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.805980 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-f8ca848e-9dc1-4cbd-be7a-bcfaa4bfd1c7 None None] MFA Rules not processed for user `8f5dc9a232374ab5bc108e9d106e0353`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:15.807199 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f8ca848e-9dc1-4cbd-be7a-bcfaa4bfd1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:15.881051 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1418/2850] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:15.892950 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c62a9-498e-43b6-a849-8cb3debc4371 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:15.895003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-dd0c62a9-498e-43b6-a849-8cb3debc4371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:15.963470 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c62a9-498e-43b6-a849-8cb3debc4371 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:15.967401 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1712533-cbdf-4405-b28f-cc66df42670b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:15.968107 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1712533-cbdf-4405-b28f-cc66df42670b service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:15.968389 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1712533-cbdf-4405-b28f-cc66df42670b service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:15.968675 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1712533-cbdf-4405-b28f-cc66df42670b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:15.985862 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1712533-cbdf-4405-b28f-cc66df42670b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:15.989001 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1712533-cbdf-4405-b28f-cc66df42670b service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:16.000735 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1433/2851] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:15 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:30:16.171758 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca848e-9dc1-4cbd-be7a-bcfaa4bfd1c7 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:16.176736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:16.250576 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca848e-9dc1-4cbd-be7a-bcfaa4bfd1c7 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:16.256038 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:16.257335 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:16.257629 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:16.257917 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] PATH_INFO: `/v3/users/ccdfa3b931614da0b202b1ce00824367` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:16.259318 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccdfa3b931614da0b202b1ce00824367)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:16.261303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:16.322524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccdfa3b931614da0b202b1ce00824367'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:16.323250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccdfa3b931614da0b202b1ce00824367'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:16.332584 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccdfa3b931614da0b202b1ce00824367'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:16.346946 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccdfa3b931614da0b202b1ce00824367'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:16.347709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccdfa3b931614da0b202b1ce00824367'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:16.367846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1419/2852] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:30:16 2022] DELETE /identity/v3/users/ccdfa3b931614da0b202b1ce00824367 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:16.372909 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a0332-45d7-4620-b1bc-623a3dd057c9 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:16.373210 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a0332-45d7-4620-b1bc-623a3dd057c9 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:16.373501 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a0332-45d7-4620-b1bc-623a3dd057c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:16.404103 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-f06a0332-45d7-4620-b1bc-623a3dd057c9 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:16.404103 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f06a0332-45d7-4620-b1bc-623a3dd057c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:16.478678 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1434/2853] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:16.488303 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:16.490158 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:16.560054 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4cb11-b2a2-428f-96c8-175de3bf08cc admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:16.564230 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27573dda-5492-4b9a-bc07-4f0e7a13146b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:16.564920 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27573dda-5492-4b9a-bc07-4f0e7a13146b service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:16.565204 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27573dda-5492-4b9a-bc07-4f0e7a13146b service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:16.565481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27573dda-5492-4b9a-bc07-4f0e7a13146b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:16.576703 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27573dda-5492-4b9a-bc07-4f0e7a13146b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:16.578206 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27573dda-5492-4b9a-bc07-4f0e7a13146b service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:16.594637 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1420/2854] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:30:16.923928 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a0332-45d7-4620-b1bc-623a3dd057c9 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:16.926406 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-f06a0332-45d7-4620-b1bc-623a3dd057c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:16.939594 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:16.948426 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a0332-45d7-4620-b1bc-623a3dd057c9 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:16.953460 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:16.954729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:16.954729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:16.954960 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] PATH_INFO: `/v3/projects/453e045c5b2544d59a5b42256a194c14` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:16.957056 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=453e045c5b2544d59a5b42256a194c14)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:16.959287 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:17.016469 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453e045c5b2544d59a5b42256a194c14', 'request_id': 'req-cca5bb95-860f-4af9-9f29-26352a3c44ee'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.023104 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453e045c5b2544d59a5b42256a194c14', 'request_id': 'req-cca5bb95-860f-4af9-9f29-26352a3c44ee'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.030596 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453e045c5b2544d59a5b42256a194c14', 'request_id': 'req-cca5bb95-860f-4af9-9f29-26352a3c44ee'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.050302 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] The token cache is being invalidate because project 453e045c5b2544d59a5b42256a194c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:30:17.050711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.051743 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.053510 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1435/2855] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:30:16 2022] DELETE /identity/v3/projects/453e045c5b2544d59a5b42256a194c14 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:17.057793 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27573dda-5492-4b9a-bc07-4f0e7a13146b service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:17.068516 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27573dda-5492-4b9a-bc07-4f0e7a13146b service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:17.090135 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:17.131585 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10771d-b5f2-4987-a0eb-2ea08cf6981d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:17.132553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10771d-b5f2-4987-a0eb-2ea08cf6981d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:17.132850 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10771d-b5f2-4987-a0eb-2ea08cf6981d admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:17.133129 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10771d-b5f2-4987-a0eb-2ea08cf6981d admin admin] PATH_INFO: `/v3/users/8f5dc9a232374ab5bc108e9d106e0353` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:17.134337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10771d-b5f2-4987-a0eb-2ea08cf6981d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f5dc9a232374ab5bc108e9d106e0353)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:17.136317 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10771d-b5f2-4987-a0eb-2ea08cf6981d admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:17.184150 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ea10771d-b5f2-4987-a0eb-2ea08cf6981d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f5dc9a232374ab5bc108e9d106e0353'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.184812 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ea10771d-b5f2-4987-a0eb-2ea08cf6981d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f5dc9a232374ab5bc108e9d106e0353'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.192579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ea10771d-b5f2-4987-a0eb-2ea08cf6981d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f5dc9a232374ab5bc108e9d106e0353'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.205231 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ea10771d-b5f2-4987-a0eb-2ea08cf6981d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f5dc9a232374ab5bc108e9d106e0353'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.206058 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-ea10771d-b5f2-4987-a0eb-2ea08cf6981d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f5dc9a232374ab5bc108e9d106e0353'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.220803 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1421/2856] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:30:17 2022] DELETE /identity/v3/users/8f5dc9a232374ab5bc108e9d106e0353 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:17.551131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:17.553445 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:17.576524 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5bb95-860f-4af9-9f29-26352a3c44ee admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:17.660902 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b765efd-e54f-40ba-af62-15e023b77ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:17.661863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b765efd-e54f-40ba-af62-15e023b77ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:17.662150 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b765efd-e54f-40ba-af62-15e023b77ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:17.662455 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b765efd-e54f-40ba-af62-15e023b77ff2 admin admin] PATH_INFO: `/v3/projects/e36a328cade748a8b22fe186c7eac6d9` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:17.663754 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b765efd-e54f-40ba-af62-15e023b77ff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e36a328cade748a8b22fe186c7eac6d9)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:17.665971 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b765efd-e54f-40ba-af62-15e023b77ff2 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:17.736272 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4b765efd-e54f-40ba-af62-15e023b77ff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e36a328cade748a8b22fe186c7eac6d9', 'request_id': 'req-4b765efd-e54f-40ba-af62-15e023b77ff2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.743200 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4b765efd-e54f-40ba-af62-15e023b77ff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e36a328cade748a8b22fe186c7eac6d9', 'request_id': 'req-4b765efd-e54f-40ba-af62-15e023b77ff2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.749852 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4b765efd-e54f-40ba-af62-15e023b77ff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e36a328cade748a8b22fe186c7eac6d9', 'request_id': 'req-4b765efd-e54f-40ba-af62-15e023b77ff2'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.764459 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4b765efd-e54f-40ba-af62-15e023b77ff2 admin admin] The token cache is being invalidate because project e36a328cade748a8b22fe186c7eac6d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:30:17.764765 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4b765efd-e54f-40ba-af62-15e023b77ff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.765421 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-4b765efd-e54f-40ba-af62-15e023b77ff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:17.767092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1436/2857] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:30:17 2022] DELETE /identity/v3/projects/e36a328cade748a8b22fe186c7eac6d9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 07:30:17.772786 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c11f-00a1-4b11-9a26-2769db86b24e None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:17.773084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c11f-00a1-4b11-9a26-2769db86b24e None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:17.773362 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c11f-00a1-4b11-9a26-2769db86b24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:17.802325 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-9567c11f-00a1-4b11-9a26-2769db86b24e None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:17.803677 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-9567c11f-00a1-4b11-9a26-2769db86b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:17.899548 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1422/2858] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 07:30:17.907794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7538e-404a-483c-a2c2-581950557d35 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:17.908135 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7538e-404a-483c-a2c2-581950557d35 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:17.908482 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7538e-404a-483c-a2c2-581950557d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:17.930010 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-29c7538e-404a-483c-a2c2-581950557d35 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:17.933131 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-29c7538e-404a-483c-a2c2-581950557d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:17.940571 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1437/2859] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:17.945398 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c11f-00a1-4b11-9a26-2769db86b24e None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:17.955417 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c11f-00a1-4b11-9a26-2769db86b24e None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:17.962149 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8fbc6-bf44-43d9-bf79-29acc67c9c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:17.964842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8fbc6-bf44-43d9-bf79-29acc67c9c08 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:17.964842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8fbc6-bf44-43d9-bf79-29acc67c9c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:17.964842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8fbc6-bf44-43d9-bf79-29acc67c9c08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:17.964842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8fbc6-bf44-43d9-bf79-29acc67c9c08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:30:17.964842 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8fbc6-bf44-43d9-bf79-29acc67c9c08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:17.966608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8fbc6-bf44-43d9-bf79-29acc67c9c08 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:17.979264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1423/2860] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:30:17.984192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7538e-404a-483c-a2c2-581950557d35 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:17.994996 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7538e-404a-483c-a2c2-581950557d35 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:18.002887 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56574bc9-01ca-4057-adca-29585b956b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:18.003685 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56574bc9-01ca-4057-adca-29585b956b3f admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:18.003982 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56574bc9-01ca-4057-adca-29585b956b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:18.004397 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56574bc9-01ca-4057-adca-29585b956b3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:18.005340 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56574bc9-01ca-4057-adca-29585b956b3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:18.007789 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56574bc9-01ca-4057-adca-29585b956b3f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:18.012364 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:18.040894 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1438/2861] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:30:17 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 07:30:18.047455 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8fbc6-bf44-43d9-bf79-29acc67c9c08 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:18.058899 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8fbc6-bf44-43d9-bf79-29acc67c9c08 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:18.092244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:18.129779 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d563f9-f24b-4228-874c-6b419512b065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:18.130557 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d563f9-f24b-4228-874c-6b419512b065 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:18.130880 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d563f9-f24b-4228-874c-6b419512b065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:18.131197 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d563f9-f24b-4228-874c-6b419512b065 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:18.131963 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d563f9-f24b-4228-874c-6b419512b065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:18.134001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d563f9-f24b-4228-874c-6b419512b065 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:18.155220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1424/2862] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:30:18 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:18.160568 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56574bc9-01ca-4057-adca-29585b956b3f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:18.171638 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56574bc9-01ca-4057-adca-29585b956b3f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:18.177074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf31486-77a8-4e08-a716-13f52ed70a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:18.177753 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf31486-77a8-4e08-a716-13f52ed70a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:18.178069 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf31486-77a8-4e08-a716-13f52ed70a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:18.178350 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf31486-77a8-4e08-a716-13f52ed70a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:18.179083 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf31486-77a8-4e08-a716-13f52ed70a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:18.181003 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf31486-77a8-4e08-a716-13f52ed70a70 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:18.191388 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1439/2863] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:18.195524 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d563f9-f24b-4228-874c-6b419512b065 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:18.206504 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d563f9-f24b-4228-874c-6b419512b065 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:18.211993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829383-4ae2-485b-93fb-86c2f24e86c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:18.212709 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829383-4ae2-485b-93fb-86c2f24e86c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:18.212993 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829383-4ae2-485b-93fb-86c2f24e86c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:18.213275 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829383-4ae2-485b-93fb-86c2f24e86c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:18.214123 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6829383-4ae2-485b-93fb-86c2f24e86c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:18.215728 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6829383-4ae2-485b-93fb-86c2f24e86c9 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:18.227644 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1425/2864] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:18.232299 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf31486-77a8-4e08-a716-13f52ed70a70 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:18.244030 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf31486-77a8-4e08-a716-13f52ed70a70 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:18.248796 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4acde-76da-49f2-8b4f-0fb58231055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:18.249681 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4acde-76da-49f2-8b4f-0fb58231055e admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:18.249963 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4acde-76da-49f2-8b4f-0fb58231055e admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:18.250235 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4acde-76da-49f2-8b4f-0fb58231055e admin admin] PATH_INFO: `/v3/projects/24870aec121646afb5b93768b031d87e/users/06675b4bb7bb4d819e1bec5d789d7caa/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:18.266824 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4acde-76da-49f2-8b4f-0fb58231055e admin admin] RBAC: Authorizing `identity:create_grant(project_id=24870aec121646afb5b93768b031d87e, user_id=06675b4bb7bb4d819e1bec5d789d7caa, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:18.270092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4acde-76da-49f2-8b4f-0fb58231055e admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:18.286662 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1440/2865] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:30:18 2022] PUT /identity/v3/projects/24870aec121646afb5b93768b031d87e/users/06675b4bb7bb4d819e1bec5d789d7caa/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:18.297606 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536c35c-0b8b-4ef5-96f5-5d03916e4e97 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:18.297905 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536c35c-0b8b-4ef5-96f5-5d03916e4e97 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:18.298187 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536c35c-0b8b-4ef5-96f5-5d03916e4e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:18.332972 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-c536c35c-0b8b-4ef5-96f5-5d03916e4e97 None None] MFA Rules not processed for user `06675b4bb7bb4d819e1bec5d789d7caa`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:18.334915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-c536c35c-0b8b-4ef5-96f5-5d03916e4e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:18.417244 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1426/2866] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:18.430817 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4acde-76da-49f2-8b4f-0fb58231055e admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:18.432861 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-80a4acde-76da-49f2-8b4f-0fb58231055e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:18.491291 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4acde-76da-49f2-8b4f-0fb58231055e admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:18.494929 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902ae6f-35fa-4941-9db8-09b88d0e2d86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:18.495576 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902ae6f-35fa-4941-9db8-09b88d0e2d86 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:18.495860 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902ae6f-35fa-4941-9db8-09b88d0e2d86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:18.496161 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902ae6f-35fa-4941-9db8-09b88d0e2d86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:18.505665 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902ae6f-35fa-4941-9db8-09b88d0e2d86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:18.507093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902ae6f-35fa-4941-9db8-09b88d0e2d86 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:18.516489 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1441/2867] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:30:18.538707 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536c35c-0b8b-4ef5-96f5-5d03916e4e97 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:18.604797 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536c35c-0b8b-4ef5-96f5-5d03916e4e97 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:18.609553 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:18.610437 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:18.610717 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:18.610999 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] PATH_INFO: `/v3/users/06675b4bb7bb4d819e1bec5d789d7caa` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:18.612277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06675b4bb7bb4d819e1bec5d789d7caa)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:18.614215 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:18.656882 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:18.663427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06675b4bb7bb4d819e1bec5d789d7caa'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:18.664262 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06675b4bb7bb4d819e1bec5d789d7caa'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:18.673296 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06675b4bb7bb4d819e1bec5d789d7caa'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:18.685608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06675b4bb7bb4d819e1bec5d789d7caa'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:18.686341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06675b4bb7bb4d819e1bec5d789d7caa'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:18.700427 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1427/2868] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:30:18 2022] DELETE /identity/v3/users/06675b4bb7bb4d819e1bec5d789d7caa => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:18.705093 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be4aa8-d53f-4cad-8820-02c79309faad None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:18.705942 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be4aa8-d53f-4cad-8820-02c79309faad None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:18.706225 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be4aa8-d53f-4cad-8820-02c79309faad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:18.723379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-70be4aa8-d53f-4cad-8820-02c79309faad None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:18.724639 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-70be4aa8-d53f-4cad-8820-02c79309faad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:18.786646 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1442/2869] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:18.793846 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:18.795528 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:18.870222 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee61395-f6a7-4c26-aaa8-6b99ea4e0db4 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:18.874302 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9aaba2-c491-4573-810e-cbd635546db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:18.874958 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9aaba2-c491-4573-810e-cbd635546db9 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:18.875277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9aaba2-c491-4573-810e-cbd635546db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:18.875675 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9aaba2-c491-4573-810e-cbd635546db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:18.887707 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9aaba2-c491-4573-810e-cbd635546db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:18.889184 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9aaba2-c491-4573-810e-cbd635546db9 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:18.900233 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1428/2870] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:30:19.200684 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be4aa8-d53f-4cad-8820-02c79309faad None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:19.202141 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-70be4aa8-d53f-4cad-8820-02c79309faad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:19.227842 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be4aa8-d53f-4cad-8820-02c79309faad None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:19.233214 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac1083-3c32-4d22-bcb1-2696f1d2e328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:19.234122 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac1083-3c32-4d22-bcb1-2696f1d2e328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:19.234415 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac1083-3c32-4d22-bcb1-2696f1d2e328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:19.234755 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac1083-3c32-4d22-bcb1-2696f1d2e328 admin admin] PATH_INFO: `/v3/projects/24870aec121646afb5b93768b031d87e` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:19.235978 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac1083-3c32-4d22-bcb1-2696f1d2e328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24870aec121646afb5b93768b031d87e)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:19.237998 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac1083-3c32-4d22-bcb1-2696f1d2e328 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:19.288220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1bac1083-3c32-4d22-bcb1-2696f1d2e328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24870aec121646afb5b93768b031d87e', 'request_id': 'req-1bac1083-3c32-4d22-bcb1-2696f1d2e328'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:19.289447 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:19.295177 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1bac1083-3c32-4d22-bcb1-2696f1d2e328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24870aec121646afb5b93768b031d87e', 'request_id': 'req-1bac1083-3c32-4d22-bcb1-2696f1d2e328'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:19.302974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1bac1083-3c32-4d22-bcb1-2696f1d2e328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24870aec121646afb5b93768b031d87e', 'request_id': 'req-1bac1083-3c32-4d22-bcb1-2696f1d2e328'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:19.316146 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1bac1083-3c32-4d22-bcb1-2696f1d2e328 admin admin] The token cache is being invalidate because project 24870aec121646afb5b93768b031d87e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:30:19.316444 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1bac1083-3c32-4d22-bcb1-2696f1d2e328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:19.317145 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-1bac1083-3c32-4d22-bcb1-2696f1d2e328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:19.319187 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1443/2871] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:30:19 2022] DELETE /identity/v3/projects/24870aec121646afb5b93768b031d87e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:19.324704 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ed044-a2f0-4e7c-8c2b-f7db84edaaf4 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:19.325214 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ed044-a2f0-4e7c-8c2b-f7db84edaaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:19.325347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ed044-a2f0-4e7c-8c2b-f7db84edaaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:19.343808 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-871ed044-a2f0-4e7c-8c2b-f7db84edaaf4 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:19.345062 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-871ed044-a2f0-4e7c-8c2b-f7db84edaaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:19.409598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1429/2872] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:19.418315 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadd8b3-0f6e-40e4-a3a0-17d509ab9855 None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:19.418608 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadd8b3-0f6e-40e4-a3a0-17d509ab9855 None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:19.418893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadd8b3-0f6e-40e4-a3a0-17d509ab9855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:19.437486 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-7aadd8b3-0f6e-40e4-a3a0-17d509ab9855 None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:19.438714 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-7aadd8b3-0f6e-40e4-a3a0-17d509ab9855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:19.447305 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1444/2873] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:19.451598 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ed044-a2f0-4e7c-8c2b-f7db84edaaf4 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:19.462333 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ed044-a2f0-4e7c-8c2b-f7db84edaaf4 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:19.466998 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67503a-d68c-4b46-9b4e-61be5f70007c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:19.467664 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67503a-d68c-4b46-9b4e-61be5f70007c admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:19.467951 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67503a-d68c-4b46-9b4e-61be5f70007c admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:19.468264 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67503a-d68c-4b46-9b4e-61be5f70007c admin admin] PATH_INFO: `/v3/domains` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:19.469001 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67503a-d68c-4b46-9b4e-61be5f70007c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 07:30:19.469310 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67503a-d68c-4b46-9b4e-61be5f70007c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:19.470915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67503a-d68c-4b46-9b4e-61be5f70007c admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:19.482682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1430/2874] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jun 17 07:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 07:30:19.487178 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadd8b3-0f6e-40e4-a3a0-17d509ab9855 None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:19.496656 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadd8b3-0f6e-40e4-a3a0-17d509ab9855 None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:19.501461 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68adf435-663b-4123-a74f-65e120f31842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:19.502623 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adf435-663b-4123-a74f-65e120f31842 admin admin] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:19.502916 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adf435-663b-4123-a74f-65e120f31842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:19.503224 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adf435-663b-4123-a74f-65e120f31842 admin admin] PATH_INFO: `/v3/projects` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:19.503921 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68adf435-663b-4123-a74f-65e120f31842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:19.505880 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68adf435-663b-4123-a74f-65e120f31842 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:19.533934 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1445/2875] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jun 17 07:30:19 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:19.590667 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67503a-d68c-4b46-9b4e-61be5f70007c admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:19.601118 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67503a-d68c-4b46-9b4e-61be5f70007c admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:19.651347 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:19.664640 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95337d4-9f40-4f4a-aad5-5a755538a496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:19.665319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95337d4-9f40-4f4a-aad5-5a755538a496 admin admin] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:19.665608 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95337d4-9f40-4f4a-aad5-5a755538a496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:19.665884 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95337d4-9f40-4f4a-aad5-5a755538a496 admin admin] PATH_INFO: `/v3/users` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:19.666588 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95337d4-9f40-4f4a-aad5-5a755538a496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:19.668539 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95337d4-9f40-4f4a-aad5-5a755538a496 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:19.689440 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1431/2876] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jun 17 07:30:19 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 07:30:19.694159 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68adf435-663b-4123-a74f-65e120f31842 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:19.704657 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68adf435-663b-4123-a74f-65e120f31842 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:19.709914 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b607-7b19-4cc6-8557-f52e790abd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:19.710582 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b607-7b19-4cc6-8557-f52e790abd1f admin admin] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:19.710865 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b607-7b19-4cc6-8557-f52e790abd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:19.711190 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b607-7b19-4cc6-8557-f52e790abd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:19.711893 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1b607-7b19-4cc6-8557-f52e790abd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:19.713549 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1b607-7b19-4cc6-8557-f52e790abd1f admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:19.725094 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1446/2877] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:19.729203 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95337d4-9f40-4f4a-aad5-5a755538a496 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:19.739773 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95337d4-9f40-4f4a-aad5-5a755538a496 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:19.744535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33b8ef-0c22-4810-9f61-b65166e1dee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:19.745201 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33b8ef-0c22-4810-9f61-b65166e1dee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:19.745656 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33b8ef-0c22-4810-9f61-b65166e1dee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:19.745938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33b8ef-0c22-4810-9f61-b65166e1dee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:19.746670 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33b8ef-0c22-4810-9f61-b65166e1dee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:19.748292 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33b8ef-0c22-4810-9f61-b65166e1dee1 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:19.759818 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1432/2878] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jun 17 07:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 07:30:19.764153 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b607-7b19-4cc6-8557-f52e790abd1f admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:19.775030 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b607-7b19-4cc6-8557-f52e790abd1f admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:19.779676 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb285a2a-ba23-4eed-a6a4-db8c06500dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:19.780615 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb285a2a-ba23-4eed-a6a4-db8c06500dda admin admin] REQUEST_METHOD: `PUT` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:19.780886 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb285a2a-ba23-4eed-a6a4-db8c06500dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:19.781156 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb285a2a-ba23-4eed-a6a4-db8c06500dda admin admin] PATH_INFO: `/v3/projects/b52b7c6031a24e71b1385cd3cbfc8284/users/df29ecc715b249308bf0b432fd7a2adf/roles/ce36ba2b01744465803d0533b2a47592` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:19.796600 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb285a2a-ba23-4eed-a6a4-db8c06500dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52b7c6031a24e71b1385cd3cbfc8284, user_id=df29ecc715b249308bf0b432fd7a2adf, role_id=ce36ba2b01744465803d0533b2a47592)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:19.799794 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb285a2a-ba23-4eed-a6a4-db8c06500dda admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:19.814220 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1447/2879] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jun 17 07:30:19 2022] PUT /identity/v3/projects/b52b7c6031a24e71b1385cd3cbfc8284/users/df29ecc715b249308bf0b432fd7a2adf/roles/ce36ba2b01744465803d0533b2a47592 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:19.824902 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31a75-14cd-4869-9ca2-41b77b335c88 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:19.825198 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31a75-14cd-4869-9ca2-41b77b335c88 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:19.825482 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31a75-14cd-4869-9ca2-41b77b335c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:19.854445 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-24e31a75-14cd-4869-9ca2-41b77b335c88 None None] MFA Rules not processed for user `df29ecc715b249308bf0b432fd7a2adf`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:19.855583 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-24e31a75-14cd-4869-9ca2-41b77b335c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:19.923592 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1433/2880] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:19.935254 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb285a2a-ba23-4eed-a6a4-db8c06500dda admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:19.936999 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-cb285a2a-ba23-4eed-a6a4-db8c06500dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:20.004728 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb285a2a-ba23-4eed-a6a4-db8c06500dda admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:20.008814 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5184a48-89fd-45c8-a72e-3cadeaae11c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:20.009484 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5184a48-89fd-45c8-a72e-3cadeaae11c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:20.009772 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5184a48-89fd-45c8-a72e-3cadeaae11c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:20.010053 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5184a48-89fd-45c8-a72e-3cadeaae11c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:20.021075 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5184a48-89fd-45c8-a72e-3cadeaae11c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:20.022529 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5184a48-89fd-45c8-a72e-3cadeaae11c1 service cinder] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:20.032080 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1448/2881] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:30:22.586055 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e31a75-14cd-4869-9ca2-41b77b335c88 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:22.667272 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e31a75-14cd-4869-9ca2-41b77b335c88 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:22.673527 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:22.675349 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:22.675736 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:22.676220 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] PATH_INFO: `/v3/users/df29ecc715b249308bf0b432fd7a2adf` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:22.678263 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df29ecc715b249308bf0b432fd7a2adf)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:22.682025 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:22.748207 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df29ecc715b249308bf0b432fd7a2adf'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:22.748885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df29ecc715b249308bf0b432fd7a2adf'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:22.756194 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df29ecc715b249308bf0b432fd7a2adf'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:22.758162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:22.773277 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df29ecc715b249308bf0b432fd7a2adf'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:22.774019 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df29ecc715b249308bf0b432fd7a2adf'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:22.788756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1434/2882] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:30:22 2022] DELETE /identity/v3/users/df29ecc715b249308bf0b432fd7a2adf => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:22.793519 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186610b0-9dfb-4a89-a727-05652235749b None None] REQUEST_METHOD: `POST` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:22.794099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186610b0-9dfb-4a89-a727-05652235749b None None] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:22.794099 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186610b0-9dfb-4a89-a727-05652235749b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:22.813249 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.auth.core [None req-186610b0-9dfb-4a89-a727-05652235749b None None] MFA Rules not processed for user `0bf6e418289c4b9c81b349d812dd7d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80118) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:22.814496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-186610b0-9dfb-4a89-a727-05652235749b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:22.884334 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1449/2883] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:22.894796 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:22.896337 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:22.960661 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51140b9-84eb-49ad-abfc-896956492444 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:22.964542 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6a4e8-1e56-45cd-b190-3261d38510af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:22.965227 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6a4e8-1e56-45cd-b190-3261d38510af service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:22.965518 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6a4e8-1e56-45cd-b190-3261d38510af service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:22.965800 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6a4e8-1e56-45cd-b190-3261d38510af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:22.976494 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6a4e8-1e56-45cd-b190-3261d38510af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:22.977996 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6a4e8-1e56-45cd-b190-3261d38510af service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:22.989341 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1435/2884] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:30:23.296863 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186610b0-9dfb-4a89-a727-05652235749b None None] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:23.298379 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-186610b0-9dfb-4a89-a727-05652235749b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:23.313725 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186610b0-9dfb-4a89-a727-05652235749b None None] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:23.319799 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:23.321974 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:23.322726 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:23.323018 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] PATH_INFO: `/v3/projects/b52b7c6031a24e71b1385cd3cbfc8284` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:23.324499 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52b7c6031a24e71b1385cd3cbfc8284)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:23.326404 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:23.340050 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:23.381313 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52b7c6031a24e71b1385cd3cbfc8284', 'request_id': 'req-3daced7d-34d7-439a-848e-e289d6074c39'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:23.387603 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52b7c6031a24e71b1385cd3cbfc8284', 'request_id': 'req-3daced7d-34d7-439a-848e-e289d6074c39'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:23.396005 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52b7c6031a24e71b1385cd3cbfc8284', 'request_id': 'req-3daced7d-34d7-439a-848e-e289d6074c39'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:23.412248 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] The token cache is being invalidate because project b52b7c6031a24e71b1385cd3cbfc8284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:30:23.412545 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:23.413241 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:30:23.414870 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1450/2885] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:30:23 2022] DELETE /identity/v3/projects/b52b7c6031a24e71b1385cd3cbfc8284 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:30:32.983973 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=33,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:33.408905 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:48.033567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6a4e8-1e56-45cd-b190-3261d38510af service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:48.035060 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b4e6a4e8-1e56-45cd-b190-3261d38510af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:48.094147 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6a4e8-1e56-45cd-b190-3261d38510af service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:48.143729 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e476d-53a8-4d1e-bfd7-adbd49f2a32e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3769534c610b448db693f7c5d6133e65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:48.144499 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e476d-53a8-4d1e-bfd7-adbd49f2a32e service cinder] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:48.144787 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e476d-53a8-4d1e-bfd7-adbd49f2a32e service cinder] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:48.145064 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e476d-53a8-4d1e-bfd7-adbd49f2a32e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:48.146878 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-383e476d-53a8-4d1e-bfd7-adbd49f2a32e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:48.206975 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e476d-53a8-4d1e-bfd7-adbd49f2a32e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:48.209134 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e476d-53a8-4d1e-bfd7-adbd49f2a32e service cinder] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:48.218569 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1436/2886] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:30:51.452046 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:51.453544 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:51.514199 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daced7d-34d7-439a-848e-e289d6074c39 admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:51.517968 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150f4f6-0b88-4404-b626-39ecb2ed0eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:51.518661 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150f4f6-0b88-4404-b626-39ecb2ed0eeb service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:51.518951 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150f4f6-0b88-4404-b626-39ecb2ed0eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:51.519232 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150f4f6-0b88-4404-b626-39ecb2ed0eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:51.521074 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e150f4f6-0b88-4404-b626-39ecb2ed0eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:51.531523 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150f4f6-0b88-4404-b626-39ecb2ed0eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:51.533001 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150f4f6-0b88-4404-b626-39ecb2ed0eeb service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:51.541181 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1451/2887] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:30:51.840653 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff49a2-f79e-44cd-bb28-0a86d2d1d3c7 None None] REQUEST_METHOD: `POST` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:51.841209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff49a2-f79e-44cd-bb28-0a86d2d1d3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:51.841209 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff49a2-f79e-44cd-bb28-0a86d2d1d3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:51.859458 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.auth.core [None req-ceff49a2-f79e-44cd-bb28-0a86d2d1d3c7 None None] MFA Rules not processed for user `6a113fe9b988458d8e8898c5c9a8d02d`. Rule list: `[]` (Enabled: `True`). {{(pid=80117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 07:30:51.860711 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ceff49a2-f79e-44cd-bb28-0a86d2d1d3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:51.868211 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1437/2888] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jun 17 07:30:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 07:30:52.339490 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150f4f6-0b88-4404-b626-39ecb2ed0eeb service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:52.348512 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150f4f6-0b88-4404-b626-39ecb2ed0eeb service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:52.352329 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd8bb2-fec8-47f7-8dd1-9f25065c278b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:52.352986 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd8bb2-fec8-47f7-8dd1-9f25065c278b service neutron] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:52.353274 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd8bb2-fec8-47f7-8dd1-9f25065c278b service neutron] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:52.353551 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd8bb2-fec8-47f7-8dd1-9f25065c278b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:52.355400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e9cd8bb2-fec8-47f7-8dd1-9f25065c278b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:52.366555 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd8bb2-fec8-47f7-8dd1-9f25065c278b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:52.368202 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd8bb2-fec8-47f7-8dd1-9f25065c278b service neutron] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:52.376808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1452/2889] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:30:52.849399 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff49a2-f79e-44cd-bb28-0a86d2d1d3c7 None None] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:52.851020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-ceff49a2-f79e-44cd-bb28-0a86d2d1d3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:52.928393 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff49a2-f79e-44cd-bb28-0a86d2d1d3c7 None None] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:52.933869 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6818c-bf0c-4f53-bf5f-6e657ec0d170 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8cf02a52d354517b363c0aaf477550c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:52.934532 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6818c-bf0c-4f53-bf5f-6e657ec0d170 service placement] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:52.934822 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6818c-bf0c-4f53-bf5f-6e657ec0d170 service placement] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:52.935101 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6818c-bf0c-4f53-bf5f-6e657ec0d170 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:52.936938 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-31f6818c-bf0c-4f53-bf5f-6e657ec0d170 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:52.945959 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=100 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:30:52.950838 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6818c-bf0c-4f53-bf5f-6e657ec0d170 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:52.952481 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6818c-bf0c-4f53-bf5f-6e657ec0d170 service placement] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:52.963208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1438/2890] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:30:55.636386 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd8bb2-fec8-47f7-8dd1-9f25065c278b service neutron] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:30:55.638092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-e9cd8bb2-fec8-47f7-8dd1-9f25065c278b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:55.745064 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd8bb2-fec8-47f7-8dd1-9f25065c278b service neutron] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:30:55.754541 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2a513-3b9c-48d1-8773-3c8e95ab6bb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:30:55.756376 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2a513-3b9c-48d1-8773-3c8e95ab6bb9 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:30:55.756742 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2a513-3b9c-48d1-8773-3c8e95ab6bb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:30:55.757035 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2a513-3b9c-48d1-8773-3c8e95ab6bb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:30:55.759239 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-8ef2a513-3b9c-48d1-8773-3c8e95ab6bb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:30:55.775496 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2a513-3b9c-48d1-8773-3c8e95ab6bb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:30:55.777175 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2a513-3b9c-48d1-8773-3c8e95ab6bb9 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:30:55.792125 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1453/2891] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:30:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:31:02.959105 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=3 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:31:04.747120 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6818c-bf0c-4f53-bf5f-6e657ec0d170 service placement] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:31:04.748826 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-31f6818c-bf0c-4f53-bf5f-6e657ec0d170 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:31:04.765658 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6818c-bf0c-4f53-bf5f-6e657ec0d170 service placement] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:31:04.771162 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155247f4-4890-41b8-a06c-b9cc2d38d52c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:31:04.771844 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155247f4-4890-41b8-a06c-b9cc2d38d52c service nova] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:31:04.772171 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155247f4-4890-41b8-a06c-b9cc2d38d52c service nova] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:31:04.772454 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155247f4-4890-41b8-a06c-b9cc2d38d52c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:31:04.774356 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-155247f4-4890-41b8-a06c-b9cc2d38d52c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:31:04.788579 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155247f4-4890-41b8-a06c-b9cc2d38d52c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:31:04.790226 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155247f4-4890-41b8-a06c-b9cc2d38d52c service nova] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:31:04.805136 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1439/2892] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:31:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:31:05.784607 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=66 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:31:08.998049 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2a513-3b9c-48d1-8773-3c8e95ab6bb9 service nova] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:31:09.010270 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2a513-3b9c-48d1-8773-3c8e95ab6bb9 service nova] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:31:09.015959 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac74c3-00ec-4f66-b5e9-1ca61075aa29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3baa16824f40f3a4c783e210b262f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:31:09.016662 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac74c3-00ec-4f66-b5e9-1ca61075aa29 service nova] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:31:09.016947 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac74c3-00ec-4f66-b5e9-1ca61075aa29 service nova] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:31:09.017228 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac74c3-00ec-4f66-b5e9-1ca61075aa29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:31:09.019048 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b2ac74c3-00ec-4f66-b5e9-1ca61075aa29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:31:09.032564 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac74c3-00ec-4f66-b5e9-1ca61075aa29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:31:09.034184 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac74c3-00ec-4f66-b5e9-1ca61075aa29 service nova] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:31:09.045874 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1454/2893] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:31:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:31:14.799078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=6 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:31:19.040807 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=6 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:32:49.207651 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155247f4-4890-41b8-a06c-b9cc2d38d52c service nova] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:32:49.209084 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-155247f4-4890-41b8-a06c-b9cc2d38d52c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:32:49.270682 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155247f4-4890-41b8-a06c-b9cc2d38d52c service nova] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:32:49.274949 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfb22f-7e7b-4c75-b60a-c4cb78622f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e0797679c07454abc867cd7b30c8b56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:32:49.275594 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfb22f-7e7b-4c75-b60a-c4cb78622f31 service neutron] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:32:49.275881 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfb22f-7e7b-4c75-b60a-c4cb78622f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:32:49.276192 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfb22f-7e7b-4c75-b60a-c4cb78622f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:32:49.278078 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e9dfb22f-7e7b-4c75-b60a-c4cb78622f31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:32:49.390915 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dfb22f-7e7b-4c75-b60a-c4cb78622f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:32:49.392446 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dfb22f-7e7b-4c75-b60a-c4cb78622f31 service neutron] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:32:49.414940 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1440/2894] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:32:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:32:51.082192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac74c3-00ec-4f66-b5e9-1ca61075aa29 service nova] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:32:51.084189 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-b2ac74c3-00ec-4f66-b5e9-1ca61075aa29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:32:51.176579 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac74c3-00ec-4f66-b5e9-1ca61075aa29 service nova] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:32:51.181729 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1152b8-f636-48a6-9899-cf4a24b42ecb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8cf02a52d354517b363c0aaf477550c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:32:51.182400 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1152b8-f636-48a6-9899-cf4a24b42ecb service placement] REQUEST_METHOD: `GET` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:32:51.182808 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1152b8-f636-48a6-9899-cf4a24b42ecb service placement] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:32:51.183092 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1152b8-f636-48a6-9899-cf4a24b42ecb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:32:51.185209 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-1e1152b8-f636-48a6-9899-cf4a24b42ecb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:32:51.199252 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1152b8-f636-48a6-9899-cf4a24b42ecb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:32:51.200896 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1152b8-f636-48a6-9899-cf4a24b42ecb service placement] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:32:51.213667 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1455/2895] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:32:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:32:55.785259 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfb22f-7e7b-4c75-b60a-c4cb78622f31 service neutron] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:32:55.787249 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-e9dfb22f-7e7b-4c75-b60a-c4cb78622f31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:32:55.805868 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfb22f-7e7b-4c75-b60a-c4cb78622f31 service neutron] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:32:55.812885 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:32:55.814249 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:32:55.814685 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:32:55.815114 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] PATH_INFO: `/v3/users/6a113fe9b988458d8e8898c5c9a8d02d` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:32:55.816962 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a113fe9b988458d8e8898c5c9a8d02d)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:32:55.820756 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:32:55.878530 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a113fe9b988458d8e8898c5c9a8d02d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:55.879208 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a113fe9b988458d8e8898c5c9a8d02d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:55.884506 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:32:55.889183 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a113fe9b988458d8e8898c5c9a8d02d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:55.905049 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a113fe9b988458d8e8898c5c9a8d02d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:55.905740 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a113fe9b988458d8e8898c5c9a8d02d'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:55.921319 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1441/2896] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:32:55 2022] DELETE /identity/v3/users/6a113fe9b988458d8e8898c5c9a8d02d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:32:56.111085 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1152b8-f636-48a6-9899-cf4a24b42ecb service placement] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:32:56.127585 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1152b8-f636-48a6-9899-cf4a24b42ecb service placement] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:32:56.189001 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:32:56.189915 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:32:56.190165 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:32:56.190446 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] PATH_INFO: `/v3/projects/d2b7ae6a418640629020f39ac455d075` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:32:56.191692 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2b7ae6a418640629020f39ac455d075)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:32:56.193677 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:32:56.250642 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2b7ae6a418640629020f39ac455d075', 'request_id': 'req-463097a4-8eea-4f91-812f-63078a9c2fee'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.256213 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2b7ae6a418640629020f39ac455d075', 'request_id': 'req-463097a4-8eea-4f91-812f-63078a9c2fee'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.263136 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2b7ae6a418640629020f39ac455d075', 'request_id': 'req-463097a4-8eea-4f91-812f-63078a9c2fee'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.277282 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] The token cache is being invalidate because project d2b7ae6a418640629020f39ac455d075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:32:56.277567 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.278222 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.279797 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1456/2897] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:32:56 2022] DELETE /identity/v3/projects/d2b7ae6a418640629020f39ac455d075 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:32:56.283821 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:32:56.285635 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:32:56.299168 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1da1ed-9ec1-4b18-a2d7-328cee0c3bb0 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:32:56.357789 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:32:56.358712 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:32:56.358982 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:32:56.359250 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] PATH_INFO: `/v3/users/21d37c60639641d4a2e39141346d272e` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:32:56.360479 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21d37c60639641d4a2e39141346d272e)` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:32:56.362405 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:32:56.418413 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21d37c60639641d4a2e39141346d272e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.419431 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21d37c60639641d4a2e39141346d272e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.426783 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21d37c60639641d4a2e39141346d272e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.439692 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21d37c60639641d4a2e39141346d272e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.440298 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.notifications [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21d37c60639641d4a2e39141346d272e'} {{(pid=80117) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.454904 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1442/2898] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jun 17 07:32:56 2022] DELETE /identity/v3/users/21d37c60639641d4a2e39141346d272e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:32:56.626634 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] Authenticating user token {{(pid=80118) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:32:56.628194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.fernet_utils [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80118) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:32:56.642015 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463097a4-8eea-4f91-812f-63078a9c2fee admin admin] Validating token access rules against request {{(pid=80118) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:32:56.648552 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:32:56.708757 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f2625-6e66-487c-a046-2c4219f35862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf6e418289c4b9c81b349d812dd7d25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7107699553c04a40be061877b81300e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80118) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:32:56.709529 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f2625-6e66-487c-a046-2c4219f35862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:32:56.709813 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f2625-6e66-487c-a046-2c4219f35862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:32:56.710192 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f2625-6e66-487c-a046-2c4219f35862 admin admin] PATH_INFO: `/v3/projects/497160218e904aff92a91866c5ba3710` {{(pid=80118) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:32:56.711350 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f2625-6e66-487c-a046-2c4219f35862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=497160218e904aff92a91866c5ba3710)` {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:32:56.713293 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f2625-6e66-487c-a046-2c4219f35862 admin admin] RBAC: Authorization granted {{(pid=80118) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:32:56.760383 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-899f2625-6e66-487c-a046-2c4219f35862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '497160218e904aff92a91866c5ba3710', 'request_id': 'req-899f2625-6e66-487c-a046-2c4219f35862'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.766635 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-899f2625-6e66-487c-a046-2c4219f35862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '497160218e904aff92a91866c5ba3710', 'request_id': 'req-899f2625-6e66-487c-a046-2c4219f35862'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.773194 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-899f2625-6e66-487c-a046-2c4219f35862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '497160218e904aff92a91866c5ba3710', 'request_id': 'req-899f2625-6e66-487c-a046-2c4219f35862'} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.785682 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-899f2625-6e66-487c-a046-2c4219f35862 admin admin] The token cache is being invalidate because project 497160218e904aff92a91866c5ba3710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80118) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 07:32:56.785973 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-899f2625-6e66-487c-a046-2c4219f35862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.786598 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG keystone.notifications [None req-899f2625-6e66-487c-a046-2c4219f35862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80118) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 07:32:56.788711 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: [pid: 80118|app: 0|req: 1457/2899] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jun 17 07:32:56 2022] DELETE /identity/v3/projects/497160218e904aff92a91866c5ba3710 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 07:33:06.451062 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=63,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:33:06.782480 dv-cinder-834822-4-iscsi devstack@keystone.service[80118]: DEBUG dbcounter [-] [80118] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80118) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 07:37:53.086062 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] Authenticating user token {{(pid=80117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 07:37:53.087497 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:37:53.156571 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559b4a3-3ce6-4982-9071-2c4d08bfe0f3 admin admin] Validating token access rules against request {{(pid=80117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 07:37:53.216020 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5314efd-362e-4a98-a7e5-5b1ea6e7cb38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8cf02a52d354517b363c0aaf477550c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4ec9b09302d4c918ef638d794468ba3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 07:37:53.216710 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5314efd-362e-4a98-a7e5-5b1ea6e7cb38 service placement] REQUEST_METHOD: `GET` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 07:37:53.216995 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5314efd-362e-4a98-a7e5-5b1ea6e7cb38 service placement] SCRIPT_NAME: `/identity` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 07:37:53.217271 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5314efd-362e-4a98-a7e5-5b1ea6e7cb38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 07:37:53.219068 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.fernet_utils [None req-b5314efd-362e-4a98-a7e5-5b1ea6e7cb38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 07:37:53.230567 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5314efd-362e-4a98-a7e5-5b1ea6e7cb38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 07:37:53.232178 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5314efd-362e-4a98-a7e5-5b1ea6e7cb38 service placement] RBAC: Authorization granted {{(pid=80117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 07:37:53.242535 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: [pid: 80117|app: 0|req: 1443/2900] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jun 17 07:37:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 07:38:03.237794 dv-cinder-834822-4-iscsi devstack@keystone.service[80117]: DEBUG dbcounter [-] [80117] Writing DB stats keystone:SELECT=58 {{(pid=80117) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}